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ABSTRACT 



This monograph, part of a series on student assessment, 
reports on the results of a national survey that examined institutional 
support for undergraduate student assessment. It provides a national profile 
of student assessment initiatives at comprehensive institutions and compares 
it to assessment practices and support patterns found in all types of 
postsecondary institutions. The study also offers advice for administrators, 
faculty, and staff in comprehensive institutions who are designing student 
assessment approaches and support processes that are likely to foster 
institutional improvement. Also included is the survey instrument used to 
gather study data, which institutions can use to examine patterns for student 
assessment on their own campuses. The monograph is organized in nine 
sections: (1) "Introduction and Overview"; (2) "Perspectives on Student 

Assessment in Higher Education"; (3) "The Influence of External Groups"; (4) 
"Approaches to Student Assessment"; (5) "Organizational and Administrative 
Support Patterns"; (6) "Assessment Management Practices and Policies"; (7) 
"Uses and Impacts of Student Assessment"; (8) "Key Relationships"; and (9) 
"Executive Summary: Research Results and Recommendations." Appended is the 
survey instrument, the "Inventory of Institutional Support for Student 
Assessment for Comprehensive Institutions, " and information on the 
construction and content of variables. (Contains 43 references.) (SM) 
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1. INTRODUCTION AND OVERVIEW 



This monograph reports the results of a national research survey that 
examines institutional support for undergraduate student assessment. It 
provides a national profile of student assessment initiatives at comprehensive 
institutions; advice for administrators, faculty, and staff who are designing 
student assessment approaches and support processes that are likely to foster 
institutional improvement; aiul an instrument that institutions can use to 
examine patterns for student assessment on their own campuses. 

This report is a result of the National Center for Postsecondary Improvement (NCP1) 
research program examining organizational and administrative support for student assessment in 
postsecondary institutions. In 1997-98, NCPI researchers conducted a major literature review- 
followed by a national survey on institutional support for student assessment. While other 
dissemination efforts have focused on a research audience, this report is specifically written for 
administrators, faculty and staff in comprehensive colleges and universities w'ho are involved with 
student assessment on their campuses. 

Purpose of the Monograph 

Administrators and faculty leaders have little credible and verifiable evidence to guide their 
planning and decision making regarding student assessment. The literature on student assessment 
offers many descriptions of student assessment practices at a variety of postsecondary institutions 
(Banta & Associates, 1993; Banta, Lund, Black, & Oblander, 1996) and a number of limited 
surveys have collected information concerning student assessment measures and methods used by 
focused groups of institutions (Cowart, 1990; El-Khawas, 1988, 1989. 1990. 1991. 1992. 1993. 
1995, 1996; Johnson, Prus, Andersen, & El Khawas, 1991). There has been comparatively less 
consideration of how colleges and universities can develop an effective institutional strategy for 
assessment or of organizational and administrative practices that support internal engagement in 
assessment and enhance the use of student assessment data. Guidelines for planning and 
implementing student assessment efforts are available (American Association for Higher Education 
[AAHE], 1992; Ewell, 1988a, 1988b; Rossman & El-Khawas, 1987) but largely missing is any 
systematic examination of the relationships among various ways institutions have approached and 
supported student assessment and the likelihood of their reaping positive impacts from student 
assessment efforts (Banta et al., 1996; Ewell, 1988b; Gray & Banta, 1997). Furthermore, there 
has been little research focused specifically on student assessment within comprehensive 
institutions. 

Understanding the national profile of undergraduate assessment activities and support 
patterns found in comprehensive institutions is important for several reasons. First, it is helpful for 
institutions to know what their peers are doing in relation to student assessment. How are other 
institutions responding to external demands for student assessment? What approaches to student 
assessment are they using? How are they supporting and promoting student assessment institution- 
wide and through their policies and practices? How are they using student assessment data? What 
impacts has student assessment had on faculty as well as institutional and student performance? 
Finally, if student assessment activities do make a difference in terms of improved teaching and 
learning on campuses, how does that happen? What external influences, institutional approaches, 
and organizational and administrative support practices encourage internal involvement in student 
assessment and positive impacts from student assessment data? 

This monograph will answer these questions through accomplishing three primary 
objectives. First, it will provide a national profile of current student assessment practices and 
institutional support patterns within comprehensive colleges and universities and compare this to 



assessment practices and support patterns found in all types of postsecondarv institutions. A 
second major objective is to provide advice for administrators, faculty, and staff in comprehensive 
institutions on designing student assessment approaches and support processes that arc likelv to 
foster institutional improvement. Finally this report includes the survey instrument used to gather 
data for this study in Appendix A. Institutions can use this inventor}' to examine patterns for 
student assessment on their own campuses. 

We will discuss the student assessment methods used by institutions in this report, but it is 
not our intent to examine the specific instruments used or the measurement issues associated with 
their use. For a comprehensive review of the literature on student assessment in postsecondarv 
institutions, readers are referred to Improving Organizational and Administrative Support for 
Student Assessment: A Review of the Research Literature (Peterson. Einarson. Trice. & Nichols. 
1997). The data from this survey have been summarized in several forms including numerous 
conference papers and journal articles and a technical report entitled Institutional Support for 
Student Assessment: Methodology and Results of a National Survey (Peterson. Einarson. 
Augustine, & Vaughan, 1999). Readers who are interested in the statistical analyses conducted on 
the student assessment data should consult the technical report. While this report relies on those 
statistical analyses, it will not provide extensive statistical details. 

Definition of Student Assessment 

Our focus in this report is on undergraduate student assessment from an institutional 
perspective. We are interested in what approach institutions use to assess student performance, 
how institutions are organized to promote and support student assessment, and how they use 
student assessment data to improve student, faculty and institutional performance. 

In this monograph, we use the term student assessment to refer to activities other than 
traditional end-of-course grading that are used to measure a diverse array of dimensions of student 
performance or development. Institutions may decide to engage in student assessment for a variety 
of reasons or differing purposes. Cognitive, affective and behavioral dimensions of student 
performance and development may be assessed. Assessment efforts may be directed toward 
students as they enter the institution, during their enrollment or after they exit from the institution. 
Student assessment may use a variety of methods and may be planned and executed centrally 
within institutions or in a decentralized manner through the actions of individual academic units. 
This report will address all these dimensions of student assessment. 

Preview of the Monograph 

This report is presented in nine sections. Following this introduction, the second section, 
perspectives on student assessment, provides a brief recent history of the student assessment 
movement in general and in comprehensive institutions specifically. The conceptual framework of 
institutional support for student assessment that was developed in the literature review (Peterson et 
al., 1997) and guided the development of the survey instrument is then presented. Section two 
ends with a brief summary of the methods used to conduct the research on institutional support for 
student assessment. 

Sections three through seven present the results of a national survey of institutional support 
for student assessment. These sections parallel the domains in the framework. In each section, 
information is presented for both comprehensive institutions and all institutions that responded to 
the study. Section three focuses on institutions’ approaches to student assessment. This section is 
followed by a summary of the institution-wide support patterns evident in the study. Section five 
focuses on the influence of external groups, such as state agencies and accreditors, on institutional 
student assessment patterns. In this section, the influence of states on institutional student 
assessment activity is considered for public institutions only. Section six discusses assessment 



management practices and policies used by institutions to promote or assure the use ot' student 
assessment information. Section seven reports on the institutional uses and impacts of student 
assessment. 

Section eight is based on the statistical analyses conducted on the national survey data. It 
presents the findings of three sets of key relationships examined in the study. The first focuses on 
the influence of external groups on institutional approaches to student assessment. The second 
examines the relationship of external influences, assessment approaches, institution-wide support 
patterns, and assessment management practices and policies to the institutional uses of student 
assessment. The third analysis examines the relationship of external influences, institutional 
context, assessment approaches, institution-wide support patterns, and assessment management 
practices and policies to the impacts of engaging in student assessment. Section nine presents a 
summary of our research findings and related recommendations for specific assessment activities 
within comprehensive institutions. It concludes with general guidelines for institutions to use to 
examine and redesign or plan their student assessment process and functions. 



2. PERSPECTIVES ON STUDENT ASSESSMENT IN HIGHER EDUCATION 



An overview of the development of student assessment in higher education 
and in comprehensive institutions suggests the need for the institutional 
perspective that guides this report. A guiding framework conceptualizes seven 
domains of an institution 's student assessment strategy: institutional context, 
external influences, approach adopted , institutional support patterns, 
assessment management practices and policies, assessment culture and 
climate, and institutional uses and impacts of student assessment. 

This monograph concurs with the perspective of other scholars of student assessment. We 
agree that at its best, student assessment is not simply an exercise to gather data or respond to 
accountability requirements. Rather, an effective student assessment approach is one that gathers 
information about selected aspects of students' characteristics, achievements and experiences and 
uses this information to shape institutional policies, processes, and practices in ways that lead to 
improved student performance and institutional functioning (AAHE, 1992; Banta & Associates, 
1993; Ewell, 1984, 1987c, 1988b; Jacobi, Astin & Ayala, 1987). 

Growth of Student Assessment 

The assessment of student performance in higher and postsecondary education is not a new 
concept or phenomenon. The first College Board examination designed to assess student learning 
outcomes on a national scale was administered in 1901. The ensuing decades are marked by events 
that reflect a growing concern with assessing college student performance, such as: the emergence 
of a regional accreditation focus on student assessment; the establishment of university-based and 
national testing centers; and the development of broadened taxonomies of student outcomes 
(Resnick & Goulden, 1987; Sims, 1992). However, these earlier developments pale in comparison 
to the emergence in the mid 1980s of student assessment as an important focus of educational 
policy at the national, state and institutional level — a focus that continues today. 

This heightened interest in student assessment is the result of many broad forces. In the 
larger societal and political arena, concerns about consumer protection, the rising costs of 
education, the training and human resource needs for state and regional economic development, 
and fiscal pressures on state and federal government all have shaped public interest in the 
educational contribution of higher education institutions. At the institutional level, the challenges of 
expansion in the 1950s and 60s, the enrollment and financial constraints in the 1970s, and the new 
educational demands in the 1980s have shifted the managerial focus of performance by higher 
education institutions from resource adequacy, to efficiency, to effectiveness, to broader concerns 
for academic and institutional quality. Consequently, over the past decade and a half the 
assessment and improvement of student performance has been the focus of much of this discussion 
and of many efforts both external to and within colleges and universities. 

A variety of specific activities at the national level — reports, guidelines, legislation, 
educational goals, and funding — have been credited with providing the initial stimulus for the 
student assessment movement in higher education in the last decade and a half. A series of national 
reports that critically examined the quality of education were published in the mid 1980s. The first, 
A Nation at Risk: The Imperative for Educational Reform (National Commission on Excellence in 
Education, 1983), prompted calls for reform in elementary and secondary education. The 
following year Involvement in Learning: Realizing the Potential of American Higher Education 
(National Institute of Education, 1984) addressed the conditions required for improving the quality 
of undergraduate education. Institutions were given three recommendations: set high expectations 
for student learning, actively involve students in learning, and develop an institutionalized process 
for assessing student learning. Subsequent reports such as To Reclaim a Legacy: A Report on the 



Humanities in Higher Education (Bennett. 1984) and Integrity in the College Curriculum: A Report 
to the Academic Community (Association of American Colleges. 1985) continued this focus on 
issues of undergraduate education quality and assessment. In 1988 the U.S. Department of 
Education revised its Criteria for Recognition of Accrediting Agencies, stipulating that agencies 
must. require educational institutions and programs to (1) clearly specify their educational objectives 
and (2) conduct student assessment to determine whether they were achieving these standards. In 
addition, the enactment of legislation such as the "Student Right to Know and Campus Security 
Act" and "Ability to Benefit" legislation (Education Commission of the States. 1991). adoption of 
the National Education Goals (Lenth, 1993, 1996; Nettles. 1995). and provision of federal funds 
for the development of institutions’ assessment projects (Cook. 1989) contributed to the early 
momentum of the assessment movement. 

State governments, reflecting these national developments, have also played a key role in 
stimulating postsecondary student assessment. The first state-level mandate for student assessment 
appeared when the Florida state legislature directed the higher education system to develop the 
College-Level Academic Skills Test in 1982. Since then, the number of states involved in student 
assessment has increased steadily with all but four of fifty states now reporting some type of 
student assessment initiative (Cole, Nettles, & Sharp, 1997). States have varied greatly in their 
approaches to student assessment policy, but there is evidence of a general shift in state-level 
approaches over the past decade. From the mid to late 1980s state assessment initiatives generally 
emphasized institutional improvement as their primary purpose and largely permitted institutions to 
design their own assessment efforts. Since 1990, states have placed greater emphasis on student 
assessment as a means of responding to external demands for accountability and have increasingly 
elected to mandate the content and form of institutions' assessment approaches. 

Regional accreditation associations also have emerged as an important influence on student 
assessment. Regional accrediting associations vary in the length of time they have been involved in 
student assessment, their specific reporting requirements, and the range of assessment-related 
services provided to member institutions (Cole et al., 1997). Since the federal government revised 
its criteria for recognizing accrediting agencies in 1988, all six regional accreditation associations 
have required member institutions to undertake and document some form of student assessment 
activity. In turn, a growing number of institutions have reported that accreditation requirements are 
an important reason for deciding to engage in and increase their student assessment efforts (El- 
Khawas, 1995; Johnson et al., 1991). 

Since the mid 1980s the number of postsecondary institutions engaged in some form of 
student assessment activity has steadily increased (El-Khawas, 1988, 1990, 1995). However, by 
1990 only a small proportion of institutions had embarked on comprehensive student assessment 
programs (El-Khawas, 1990; Hexter & Lippincott, 1990; Johnson et al., 1991). Fewer still have 
reported achieving observable impacts from their student assessment efforts (Astin, 1991; 

Hutchings & Marchese, 1990). So it seems that, while many colleges and universities are investing 
faculty and administrative time and effort in student assessment, few are benefiting from its 
potential to improve student and institutional performance. 

Characteristics of Comprehensive Institutions 

Institutional type is an important dimension to consider when examining the approach an 
institution takes toward student assessment. To date, there has been little consideration of 
differences in effective organizational and administrative practices for promoting student 
assessment within specific types of institutions. Yet effective strategies and practices for planning, 
promoting and implementing undergraduate student assessment vary according to the type of 
postsecondary institution being considered — for example, what works in promoting student 
assessment in comprehensive institutions differs from what works in research institutions. In our 
national study of institutional support for student assessment, we found that there were many 



statistically significant differences by institutional type on the approaches to. support for. policies 
and practices regarding, and uses and impact of student assessment reported by institutions and 
that they were typically stronger than differences by other institutional characteristics such as size 
or whether the institution is public or private. 

' The challenges comprehensive institutions face as they develop student assessment 
programs are quite different from those faced by other institutions. These differences stem, in part, 
from the institutional mission and curricular focus, governance structure, faculty roles and 
responsibilities, and student clientele typically associated with comprehensive colleges and 
universities. 

More than other types of postsecondary institutions, comprehensive colleges and 
universities embrace a diffuse institutional mission. They function as both teaching- and research- 
oriented institutions. They give strong emphasis to undergraduate and graduate education and their 
curriculum spans a broad range from liberal arts to professional education programs. These dual 
emphases on teaching and research and on graduate and undergraduate education may affect the 
extent to which undergraduate student assessment is viewed as an institutional priority by 
administrators and faculty members alike, and so may influence their commitment to and 
involvement in assessment efforts. Building internal support for undergraduate student assessment 
may be easier to accomplish in comprehensive institutions than in research institutions, but more 
difficult than in institutions that have a clearer focus on undergraduate teaching, such as liberal arts 
and associate of arts colleges. 

The distribution of decision-making power between administrators and faculty members 
within these institutions often falls midway between these two groups and is less clear than in other 
institutional types. Faculty have less decision-making authority than their peers in doctoral and 
research institutions and selective liberal arts colleges but more than that of faculty in associate of 
arts and less selective liberal arts colleges. These differences in governance suggest that effective 
patterns of organizational and administrative support for student assessment in comprehensive 
institutions will be configured differently from those found in other types of institutions. 

Institutions with an administratively-driven governance structure, such as associate of arts 
institutions, may be able to effectively use organizational policies to direct faculty involvement in 
assessment efforts while those in which faculty have considerably more professional autonomy 
and decision-making power, such as research institutions, may need to emphasize faculty 
leadership in planning and implementing assessment activities. Given their position in the mid- 
range of faculty and administrative decision-making power, comprehensive institutions may need 
to use a mix of administratively- and faculty-driven policies and practices to effectively promote 
student assessment. 

Faculty in comprehensive institutions are expected to focus on both research and teaching 
responsibilities. On the positive side, faculty members’ focus on teaching may mean they will be 
receptive to student assessment techniques that will lead to improving the teaching and learning 
process while their familiarity with conducting research may make them quite comfortable with 
designing and implementing assessment methods and analyzing the resulting data. Conversely, it 
could be that faculty willingness to engage in student assessment is hampered in comprehensive 
institutions by their competing responsibilities for teaching and research. 

The characteristics of the student bodies at comprehensive institutions are also likely to be 
different from those at two-year and other types of four-year institutions. Comprehensive 
institutions may attract students who are less academically prepared than those who enroll in 
research, doctoral and private liberal arts institutions but more academically prepared than students 
in associate of arts institutions. Compared to other types of institutions, comprehensive institutions 
fall in the mid-range of the proportion of commuter versus residential, part-time versus full-time, 
and adult versus traditional age students enrolled. Finally, given their dual focus on undergraduate 



and graduate education, comprehensive institutions attract students with varying educational goals. 
This rather diverse student profile must be taken into account when developing assessment plans 
and activities. 

Unfortunately, there has been little research focused specifically on student assessment 
within comprehensive institutions. Although the existing literature includes some single- and multi- 
institution case studies of student assessment, it provides limited information concerning the 
current assessment activities conducted throughout the country by comprehensive colleges and 
universities. The few available studies offer conflicting evidence of the extent to which these 
institutions have been involved in undergraduate student assessment (El-Khawas & Knopp. 1996; 
Gentemann & Rogers, 1987; Scott, 1991). We did not find research that examined the relationship 
of student assessment approaches and institutional support practices to using and achieving impacts 
from student assessment in comprehensive colleges and universities. 

A Framework of Institutional Support for Student Assessment 

An institution’s organizational and administrative context can be a powerful source of 
influence on administrators’, faculty members' and students’ behaviors and perceptions and can 
shape their views of what the institution values and which activities are important to engage in. 
Colleges and universities purposefully create a variety of student assessment approaches and 
structures, processes, policies and practices to support and implement student assessment. 
Institutions have varied greatly in their approaches to assessing student performance and the ways 
in which they have organized to support their assessment efforts, a finding that is hardly surprising 
given the great diversity and relative autonomy of colleges and universities in the American 
postsecondary system. Yet there is little systematic evidence available concerning the specific 
manner in which postsecondary institutions have organized and administered to support student 
assessment and the differential results of these efforts. For these reasons, we have chosen to 
examine student assessment from a research-based and institutional perspective. 

An extensive literature review identified what is currently known about the organizational 
and administrative context for student assessment in postsecondary institutions. A detailed 
description of the literature review process and findings is available in other publications (Peterson 
et al., 1999; Peterson et al., 1997). From this review, we developed a framework of institutional 
support for student assessment. This framework, displayed in Figure 1, consists of seven 
interacting domains. It conceptualizes how institutions respond to external pressures for student 
assessment, how they approach student assessment, the organizational and administrative patterns 
they use to support student assessment, the assessment practices and policies they have adopted to 
promote student assessment, their culture and climate for student assessment, and how student 
assessment information is used by and has impacts on institutions. A seventh domain, institutional 
context, reflects the fact that these patterns probably differ significantly by institutional type, 
control and size. The culture and climate domain is not examined in this report and is the focus of 
future research activity. The other domains are briefly described below. A description of the 
variables used in this study, by domain, is provided in Appendix B. 

External Influences on Student Assessment . A variety of external constituencies have 
played an important role in initiating and shaping student assessment efforts within postsecondary 
institutions. In particular, state-level initiatives and regional accreditation associations may exert 
strong direct influences through their requirements for student assessment. The business 
community, private foundations, and professional higher education associations may serve as 
weaker sources of influence — whether as participants in institutions' assessment efforts, as 
consultants or sources of resource materials, or as providers of funds for assessment-related 
activities. National efforts appear to have largely played an indirect role in influencing institutions’ 
assessment efforts. 



Figure 1. Framework of Institutional Support for Student Assessment 
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Institutional Approach to Student Assessment . Institutional approach to student assessment 
refers to institutional decisions regarding the collection and analysis of student assessment 
information. Important dimensions along which student assessment approaches can be 
differentiated include: the type or content of student assessment measures: the extent or array of 
student assessment data collected: the timing of data collection: the instruments and methods used 
to collect student assessment data; and the analyses conducted and reported for collected data. 

Organizational and Administrative Support for Student Assessment . The literature on 
student assessment identifies five important dimensions of organizational or administrative 
behavior patterns that may support student assessment efforts and enhance the likelihood that 
assessment will contribute to improvements in institutional performance: the institution-wide 
support strategy for student assessment: administrative and governance activities used to promote 
student assessment; leadership and faculty support for student assessment; planning and 
coordination of student assessment; and procedures used to evaluate and revise student assessment 
efforts. 



Assessment Management Practices and Policies . Assessment management practices and 
policies refer to specific practices, policies, procedures, or activities intentionally devised by 
institutions to implement and support their student assessment efforts. A number of content activity 
areas of this domain have been identified as potential influences on student assessment including: 
academic resource allocation; student information systems: internal access to student information; 
distribution of assessment reports and studies; student involvement in assessment: professional 
development for student assessment; faculty evaluation and rewards: and academic planning and 
review processes. 

Institutional Uses and Impacts of Student Assessment . Ideally, institutions will use the 
information collected through student assessment processes to contribute to improvement in 
institutional and student performance. Three broad domains of institutional uses and impacts of 
student assessment information are suggested in the literature. These are the use of assessment 
information for institutional decision making, its impact on various internal constituents and 
phenomena, and its impact on the institution’s relationships with its external constituents. 

Institutions may apply assessment information to several areas of academic decision 
making such as academic planning and review, the academic mission and goals, internal resource 
allocation, and faculty evaluation and rewards. Student assessment information may have both 
internal impacts, such as stimulating faculty interest in teaching and enhancing student learning, 
and external impacts, such as influencing state funding or re-accreditation decisions. The primary 
concern of this framework is to examine the relationship of external influences, institutions’ 
assessment approaches, patterns of organizational and administrative support for assessment, and 
assessment management practices and policies to institutional uses and impacts of assessment 
information. 

Institutional Context . Broad institutional characteristics such as institutional type, control 
and size are expected to moderate external influences on assessment, the institutional approach to 
student assessment, organizational and administrative support patterns, assessment management 
practices and policies, and institutional uses and impacts of assessment information. 

Description of National Survey 

Based on our review of the student assessment literature, we developed a survey 
instrument or inventory called "Institutional Support for Student Assessment" (ISSA). The 
instrument focuses on the assessment of undergraduate students conducted by postsecondary 
institutions — not by individual faculty or academic sub-units within institutions. The instrument is 
designed as a comprehensive inventory of external influences on institutions' student assessment 



efforts, institutional approaches to student assessment, organizational and administrative support 
patterns for student assessment, assessment management practices and policies used, and 
institutional uses and impacts of student assessment information. The instrument was intended to 
assist institutions in obtaining a clear picture of their own student assessment efforts, to provide a 
national profile of undergraduate student assessment efforts in postsecondary institutions, and to 
increase current understanding of how institutions can engage in and promote student assessment 
that produces positive impacts on academic, student and institutional performance. A copv of the 
survey instrument is included in Appendix A. 

In January 1998, the ISSA instrument was mailed to the chief academic administrator at all 
postsecondary institutions recognized by the U.S. Office of Education that offer undergraduate 
programs at the associate or baccalaureate degree level, excluding proprietary and specialized 
institutions. Completed surveys were received from 1,393 of the 2.524 institutions meeting these 
criteria for an overall response rate of 55%. Comprehensive colleges and universities wereWell 
represented in survey responses. Of the 518 comprehensive institutions to whom instruments were 
mailed, 315 or 61 % completed and returned the survey. More than half of eligible public and 
private comprehensive institutions participated. Survey responses were quite evenly distributed 
across states and accrediting regions. Given this strong and diversified response rate, we are able 
to offer a representative profile of undergraduate student assessment activities undertaken in 
comprehensive colleges and universities. Throughout this report, we compare student assessment 
activities within comprehensive institutions to student assessment activities within all institutions 
responding to our survey, including comprehensive institutions. 



3. THE INFLUENCE OF EXTERNAL GROUPS 



Examines state and regional accreditation association requirements and 
external sources of support for student assessment. While both states and 
regional accrediting agencies have influenced institutions to engage, in student 
assessment, accrediting associations are reported to have greater influence 
than states on comprehensive institutions' assessment initiatives. 



State officials and regional accreditation associations have increasingly required 
postsecondary institutions to become engaged in student assessment. Together with professional 
associations and private foundations, they have also offered various services to support 
institutions’ student assessment efforts. There has been relatively little examination of institutions' 
perceptions of and experiences with these external groups. To examine this domain, our survev 
included questions concerning: state requirements for student assessment; regional accreditation 
association requirements for student assessment; and external sources of support for student 
assessment. The following sections address each of these areas in turn. 

State Role in Student Assessment 



States have varied widely in the development and content of their initiatives for 
postsecondary student assessment (Cole et al„ 1997). To examine the role of state influences in 
some detail, the survey asked for institutions’ perceptions of three dimensions of state assessment 
plans: the development process and reporting requirements of state assessment plans: the influence 
of state requirements on institutions’ assessment efforts: and state review of institutions' 
assessment plans or processes. These questions were directed to state-funded institutions only. 

Development and Reporting Requirements of State Assessment Plans 

Development . Scholars have suggested that the influence of state assessment initiatives on 
institutions’ assessment efforts may depend on whether or not institutions have had input in the 
development of the initiatives. Survey respondents were asked to indicate whether their state’s plan 
for student assessment was developed primarily by state officials, through joint consultation 
between state officials and institutional representatives or whether no state plan or requirement for 
student assessment existed. Responses from comprehensive and all institutions receiving state 
funding are displayed in Figure 2. 



Figure 2. Development of State Plan for Student Assessment 
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Half of the public comprehensive institutions in our study report that a state plan or 
requirement for student assessment exists. When state plans exist, they are more likely to have 
been developed in consultation between state and institutional officials (32%) than by state officials 
alone (16%). Public comprehensive institutions do not differ from all public institutions in their 
perceptions of the existence and development of state plans for student assessment. 

Reporting Requirements . Scholars also contend that the influence of state assessment 
initiatives will vary with the specific nature of the reporting requirements associated with these 
initiatives. What types of information are institutions required to report to state officials? We asked 
institutions with state plans for assessment which of the following types of information they were 
required to report as part of their state’s assessment plan: evidence of a student assessment plan; 
measurement of state-mandated student performance indicators; measurement of institutionally- 
developed student performance indicators; and evidence of having used student assessment 
information. Institutions checked all applicable reporting requirements. Responses from 
comprehensive and all institutions receiving state funding are displayed in Figure 3. 



Figure 3. State Reporting Requirements for Student Assessment 
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*Only institutions with a state plan for student assessment responded to this question. 



The most common reporting requirements for public comprehensive colleges and 
universities are evidence of a student assessment plan (63%) and measures of state-mandated 
student performance indicators (66%). Half of respondents say they must report on institutionally- 
devised student performance indicators (50%). A similar proportion (48%) are required to provide 
evidence of having used student assessment information. There are no major differences in the 
state reporting requirements experienced by public comprehensive institutions and all public 
institutions. This emphasis on evidence of an assessment plan and state-mandated student 
performance indicators is expected to contribute less to institutional support for student assessment 
than reporting requirements that emphasize evidence of assessment information use and 
institutionally-developed student performance indicators. 

Influence of State Assessment Requirements 

What influence have state assessment requirements had on institutions 1 undergraduate 
student assessment activities? Institutions were asked which of the following impacts on their 



student assessment activities they attributed to state assessment requirements: were an important 
reason for institution initiating undergraduate student assessment: increased institution's 
involvement in undergraduate student assessment; were not a factor in institution's undergraduate 
student assessment activities; or were a negative influence on institution's undergraduate student 
assessment activities. Institutions selected all applicable influences. Responses from public 
comprehensive institutions and all public institutions are displayed in Figure 4. 

Figure 4. Influence of State Requirements for Student Assessment on Institutions' 
Assessment Activities* 



*Only institutions with a state plan for student assessment responded to this question. 



Public comprehensive institutions most often report positive influences of state 
requirements on their student assessment activities. More than half (56%) report state requirements 
have increased their institution’s involvement in undergraduate student assessment and two-fifths 
(41%) indicate state requirements played an important role in their institution initiating student 
assessment efforts. One-third (32%) believe state requirements have had no influence on their 
institution’s student assessment efforts while only 6% indicate state requirements have been a 
negative influence. Compared to all public institutions, public comprehensive institutions are 
slightly less likely to report positive influences of state requirements on their assessment activities 
and more likely to report no influence from state requirements. 

State Review of Institutions’ Student Assessment Plans 

Occurrence of State Review . States vary in terms of whether and how they evaluate 
institutions’ student assessment plans or information (National Center for Higher Education 
Management Systems, 1996). The survey asked respondents whether state officials had reviewed 
or evaluated their institution’s student assessment plans or processes. If a state review or 
evaluation had occurred, respondents indicated whether this review had been conducted by state 
officials, external reviewers or the institution itself. Responses from public comprehensive 
institutions and all public institutions are presented in Figure 5. 

Slightly less than half of public comprehensive institutions report their undergraduate 
student assessment plan or process has been reviewed by their state higher education agency 
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(46%). Public comprehensive institutions are less likely than all public institutions to have 
undergone a state review. When a state review has occurred, it has most often been conducted bv 
state officials (36%) and less often by external reviewers (12%) or the institution itself (17%). 

Figure 5. Methods Used to Review Institutions' Student Assessment Plans or Processes* 
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*Only institutions with a state plan for student assessment responded to this question. 



State Review Criteria . Institutions that had undergone state review of their student 
assessment plan or process specified if the evaluation: reviewed the institution's student 
assessment process; compared the institution’s student performance record to its past performance; 
compared the institution’s student performance record with that of peer institutions; or compared 
the institution’s student performance record with that of other institutions in the state. Responses 
from public comprehensive institutions and all public institutions are presented in Figure 6. 

Figure 6. State Review Criteria for Institutions’ Student Assessment Plans or Processes* 
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When public comprehensive institutions have undergone a state review of their student 
assessment plan or process, this review has most often focused on the institution's student 
assessment process itself (70%). Comparisons of institutions' student performance records are less 
common. If conducted, these most often involve comparing institutions' student performance to 
their own past performance (52%) followed by comparisons with the student performance records 
at other institutions in the same state (38%) or with those of peer institutions (35%). Compared to 
all public institutions in our study, public comprehensive institutions are a little more likely to have 
their current student performance record compared with their past performance. No other 
differences in state review criteria are evident. 

Regional Accrediting Role in Student Assessment 

As noted earlier, regional accreditation agencies have been reported as important influences 
on institutions’ decisions to begin or expand their student assessment activities (El-Khawas. 1990. 
1992, 1995; Johnson et al., 1991). The majority of comprehensive institutions participating in our 
study (82%) had completed a regional accreditation review which required undergraduate student 
assessment. To further examine the role of regional accreditation associations in student 
assessment, the survey asked about the reporting requirements for regional accreditation and 
institutions’ perceptions of the influence of regional accreditation requirements on their assessment 
efforts. 

Regional Accreditation Reporting Requirements for Student Assessment 

Regional accreditation associations vary in terms of the evidence they require institutions to 
report about student assessment efforts. It may be expected that requirements to report on the use 
being made of assessment information may contribute to greater institutional impacts from 
assessment than requirements that ask merely for evidence of an assessment plan or assessment 
data. We asked institutions which of the following types of information they were required to 
report to their regional accreditation association: evidence of a student assessment plan: intended 
institutional uses of student assessment information: results of student assessment: and evidence of 
having used student assessment information. Institutions checked all applicable reporting 
requirements. Very few (4%) comprehensive institutions were unfamiliar with their regional 
accreditation requirements for student assessment. Responses from comprehensive and all 
institutions are displayed in Figure 7. 

Figure 7. Regional Accreditation Association Reporting Requirements for Student Assessment 
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The majority (94%) of comprehensive institutions must provide their regional accreditation 
association with evidence that a student assessment plan or process is in place. A large proportion 
are required to provide evidence of the use they have made (80%) or intend to make (76% ) of 
student assessment information. Slightly fewer report having to provide the results of student 
assessment (69%). The regional accreditation reporting requirements for student assessment 
reported by comprehensive institutions do not differ much from those reported by all institutions in 
our study. 

Influence of Regional Accreditation Association Requirements for Student Assessment 

What influence have regional accreditation association requirements had on institutions' 
undergraduate student assessment activities? Institutions were asked which of the following impacts 
on their student assessment activities they attributed to their regional accreditor's assessment 
requirements: were an important reason for initiating undergraduate student assessment; increased 
institution’s involvement in undergraduate student assessment; were not a factor in institution's 
undergraduate student assessment activities; or were a negative influence on institution's 
undergraduate student assessment activities. Institutions selected all applicable influences. 

Responses from comprehensive institutions and all institutions are displayed in Figure 8. 

Figure 8. Influence of Regional Accreditation Requirements for Student Assessment on 
Institutions’ Assessment Activities 
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Comprehensive institutions generally perceive regional accreditation requirements for 
student assessment as having had a positive influence on their student assessment activities. More 
than four-fifths (85%) report regional accreditation requirements have led to an increase in their 
institution’s involvement in student assessment and almost three-quarters (72%) state these 
requirements were an important reason for their institution to initiate its undergraduate student 
assessment activities. Very few feel that regional accreditation requirements have had no influence 
on their institution’s assessment efforts (8%) and fewer yet report negative consequences 
stemming from these requirements (2%). Comprehensive institutions report somewhat stronger 
regional accreditation influences on their assessment efforts than all institutions in our study. 

External Sources of Support for Student Assessment 

External constituents may influence institutions’ involvement in student assessment by 
providing funding or services to support assessment efforts. Little is known about the extent to 



which institutions have availed themselves of these external sources of support. Our survey asked 
institutions about their use of external grants and other services for improving their student 
assessment efforts. 

Receipt of External Grants for Student Assessment 

Institutions were asked if they had received grants to improve or support their student 
assessment practices from any of the following external sources: Fund for the Improvement of 
Postsecondary Education (FIPSE), other federal agencies, state incentive programs, and private 
foundations or corporate sources. Responses from comprehensive institutions and all institutions 
are presented in Figure 9. 



Figure 9. Receipt of External Grants for Student Assessment by Source 
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The majority of institutions, both comprehensive and all, have not received external grants 
for improving student assessment practices (77% and 79% respectively). When comprehensive 
institutions have received grants, these are most likely to have come from the state (10%) and 
FIPSE (9%). However, no more than 10% of comprehensive institutions report receiving 

# assessment-related grants from any of the external sources considered in this question. It is unclear 
whether this low rate of grant receipt stems from unavailability, ineligibility, or lack of awareness 
of external funding opportunities. 

Use of External Services to Support Student Assessment 

• A variety of postsecondary organizations — professional associations, regional accrediting 
associations, state-level agencies, and consortia of institutions — provide a range of services 
intended to support institutions’ student assessment efforts. These services include consultation, 
assessment conferences, training workshops, and publications or research reports on student 
assessment. For each of these four types of postsecondary organizations, the survey asked 
institutions which, if any, of these student assessment services they had used. Responses from 

0 comprehensive institutions and all institutions in our study are presented in Table 1 . 

While many comprehensive colleges and universities report that external services are not 
used or not available, they do use assessment-related services from some types of postsecondary 
organizations more than from others. They are much more likely to use services from professional 
associations and regional accreditation associations than from state agencies or institutional 

0 consortia. They are also likely to use certain types of assessment support services more than others, 

regardless of service provider. Comprehensive institutions make most frequent use of conferences 
and publications or research reports on student assessment. A smaller proportion have used training 
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workshops from one or more providers while consultation services are least likely to have been 
used. Compared to all institutions in our study, comprehensive institutions make greater use of 
publications or research reports on assessment provided by professional associations and regional 
accreditation associations. 

Table 1. Institutional Use of External Services to Support Student Assessment by Service 
Provider 

9r of Institutions Using 

Services not Consultation Assessment Training Publications or 

used or not services conferences workshops research reports 
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Summary Observations 

The previous sections have addressed the influence of external constituencies, particularly 
state agencies and regional accreditation associations, on comprehensive institutions' undergraduate 
student assessment efforts. 

State-level efforts clearly play an important role in student assessment. Half of 
comprehensive institutions in our study reported they are subject to some form of state plan or 
requirements for undergraduate student assessment and have undergone a state review of their 
student assessment plan or process. Assessment plans are more likely to have been developed 
through the joint efforts of state and institutional representatives. Based on our respondents' 
reports, there is a tendency for state reporting requirements to be centralized '(greater use of state 
than institutional indicators of student performance) and input-oriented (requiring evidence of an 
institutional assessment plan rather than institutional uses of assessment information). 
Comprehensive institutions most often perceived state requirements as having had a positive 
influence on their institution’s assessment activities. 

However, regional accreditation associations appear to be a source of even stronger and 
more positive support for comprehensive institutions’ engagement in student assessment'activities. 
Regional accreditors more often require evidence of institutions’ intended or actual uses of student 
assessment information do than state reporting requirements. This pattern of reporting requirements 
may be more conducive to building institutional support for and use of student assessment than the 
more accountability oriented approaches employed by state-level officials. And compared to state 
assessment requirements, regional accreditation requirements are more often viewed as having been 
a positive influence and less often reported as having had no influence or a negative influence on 
institutions’ assessment efforts. These findings support past research concerning the strong role of 
regional accrediting associations in promoting student assessment in postsecondary institutions. 

Finally, comprehensive institutions have made little use of external sources of support for 
student assessment. Most have not received external funds to improve their student assessment 
efforts. They are more likely to make use of conferences and publications on student assessment 
provided by professional associations and regional accreditation associations. 



4. APPROACHES TO STUDENT ASSESSMENT 



Addresses the type or content of student assessment measures used In- 
institutions, the timing of those assessments, their use of standardized 
instruments and less traditional student assessment methods, and types of 
assessment studies conducted and student performance reports produced. 
Comprehensive institutions most often collect data on students’ basic college- 
readiness skills, academic intentions, academic progress and satisfaction with 
the institution, and least often collect data on current students' higher-order 
cognitive skills, vocational skills, and personal growth, and fanner students ' - 
civic or social roles. 

Assessment approach decisions may be influenced by external mandates, institutional 
context, and domains of the organizational and administrative environment. An institution's 
assessment approach may shape the uses and impacts of assessment information. Assessment 
scholars contend that institutions that collect comprehensive student assessment information — 
those that collect data on various dimensions of student performance, at multiple points in time, 
through a variety of assessment methods — and that conduct and report a variety of analyses of 
assessment data are more likely to use and achieve positive impacts from student assessment data. 

What approaches to student assessment have comprehensive colleges and universities 
adopted? The first section of the ISS A instrument addressed the type or content of student 
assessment measures used by institutions, the timing of those assessments, their use of 
standardized instruments and less traditional student assessment methods, and types of assessment 
studies conducted and student performance reports produced. Here we examine the profile of 
student assessment approaches in comprehensive institutions and compare it to the profile for all 
types of postsecondary institutions. 

Type and Extent of Student Data Collected 

Data Collected from Current Students , 

What types of data do institutions collect from their students? Survey respondents reported 
the extent to which their institution collects ten types of student data for currently enrolled students 
and four measures for former students. Figures 10, 1 1 and 12 display mean scores for the extent to 
which comprehensive institutions and all institutions collect these data. 

Cognitive Data . Figure 10 shows the extent to which institutions collect five types of 
cognitive data: basic college-readiness skills, higher-order skills (e.g., critical thinking and 
problem solving), general education competencies, competence in major field of study (discipline- 
or program-specific knowledge), and vocational or professional skills. The latter four of these five 
types of data factored together to create a “cognitive assessment” index that we used in regression 
analyses presented in section eight. 

Comprehensive institutions conduct extensive collection of data on students’ basic college- 
readiness skills. The mean score for this measure (3.40) shows this type of data is collected for 
many to all students. This emphasis is understandable given that students entering these institutions 
may vary considerably in their academic abilities. In contrast, comprehensive institutions collect 
less data on other measures of their current students' cognitive performance. Data on students’ 
competencies in their major field and general education are collected from some to many students 
(mean scores of 2.86 and 2.61 respectively), but most comprehensive institutions collect data on 
vocational and higher-order skills from only some of their students (mean score of 2.00). They 
place greater 



Figure 10. Extent of Cognitive Data Collected on Current Students 
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emphasis on assessing major field competencies than do all institutions. Beyond this, there were 
no large differences in their extent of data collection for these cognitive measures compared to all 
institutions. 

Affective and Behavioral Data . Figure 1 1 presents five types of affective and behavioral 
data collected from currently enrolled students: personal growth and affective development, student 
experiences and involvement with the institution, student" satisfaction with the institution, students' 
academic intentions, and students’ academic progress. The first three of these items factored 
together into an “affective assessment” index that we used in regression analyses presented in 
section eight. 

Comprehensive institutions collect data on the academic progress of almost all their 
students (mean score of 3.86). This type of data is a common component of routine internal 
record-keeping. However, they are less likely to collect data on students' academic intentions 
(mean score of 2.99). It is probably easier to monitor academic progress via transcripts and other 
institutional data than 



Figure 11. Extent of Affective and Behavioral Data Collected on Current Students 



o 

c 

Q) 


C 


X 


o 


LU 


o 




CD 


o 




cd 


O 

O 


o 

o 


03 


CO 


03 


c 


Q 


CD 

CD 






| Affective Index 1 

1 = not collected; 2 = collected for some students; 3 = collected for many students; 4 = collected for all students. 




it is to collect and update information on students' goals. Comprehensive institutions ask manv 
students about their satisfaction with the institution (mean score of 2.97). Comprehensive 
institutions collect less information on aspects of their students' experiences and involvement 
within the institution, doing so for only some students (mean score of 2.68). and many do not 
collect data on students’ personal growth and affective development (mean score of 2.00). 
Compared to all institutions participating in our study, comprehensive institutions place less 
emphasis on assessing student academic intentions or expectations. 

Data Collected from Former Students 

Figure 12 displays mean scores for the extent to which comprehensive institutions and all 
institutions collect four types of data from former students: vocational or professional outcomes 
(e.g., job attainment or performance), further education (e.g., transfer, degree attainment, graduate 
study), satisfaction and experiences with the institution after leaving, and civic or social roles (e.g.. 
political, social or community involvement). These latter three items factored together to create a " 
“post-college” index used in regression analyses presented in section eight. 

Figure 12. Extent of Data Collected on Former Students 
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1 = not collected: 2 = collected for some students; 3 = collected for many students: 4 = collected for all students. 



Comprehensive institutions report moderately extensive collection of data on three 
dimensions of former students' performance or development. Measures of vocational outcomes, 
further education and satisfaction with the institution are collected from some to many of their 
former students (mean scores of 2.75, 2.71 and 2.75 respectively). However, they are 
comparatively less likely to collect data concerning former students’ civic or social roles, doing so 
for only some students (mean score of 2.08). This lower emphasis on civic and social outcomes is 
likely attributable to the evaluation criteria applied to these institutions by a variety of 
constituencies. Prospective students and their parents, employers, accreditors and state-level 
officials are more likely to evaluate comprehensive institutions’ performance on the basis of 
whether or not graduates attain field-related employment, are accepted into doctoral and research 
universities for further study, and are satisfied with their educational experiences than on 
graduates’ citizenship skills and activities. 

The profile of data collection from former students by comprehensive institutions closely 
resembles the profile reported by respondents from all types of institutions. Comprehensive 
institutions do collect more extensive data concerning former students’ civic and social roles. They 
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also collect more extensive on former students' satisfaction than do all other types of institutions 
but this difference is not of great practical significance. 

Timing of Data Collection 

Researchers have stressed that it is particularly important for institutions to collect student 
assessment data at varying points in time in order to assure that students are meeting their academic 
goals and that the college experience is “value-added.” When are comprehensive institutions 
collecting student assessment data? Table 2 presents the timing of data collection efforts for each of 
nine measures of student performance for comprehensive and all institutions. Institutional 
respondents told us whether they collect data for each measure at entry, during students’ 
enrollment, and/or at exit. Respondents could choose all three of these options for each type of 
data. 



Table 2, Institutions 
Data at Entry, While 


Collecting Cognitive 
Student is Enrolled, 


, Affective, and Behavioral Student 
and at Exit 


Assessment 


Data Collected 




9c Institutions Collecting Data 




At Entry 
Comp All 


While Enrolled 
Comp All 


At Exit 
Comp All 


Cognitive Data 








Basic Skills 


96 97 


23 21 


7 8 


Higher Order Skills 


36 32 


70 66 


34 33 


General Education 


29 32 


73 64 


33 36 


Major Competence 


7 9 


63 62 


12 64 


Vocational Skills 


7 10 


69 67 


67 60 


Affective and Behavioral 








Data 








Personal Growth 


36 32 


68 67 


44 41 


Student Involvement 


6 7 


73 72 


55 55 


Student Satisfaction 


6 7 


76 71 


67 65 


Academic Intentions 


86 92 


41 36 


19 19 



Timing of Collecting Cognitive Data 

With the exception of basic skills data, most cognitive data are collected either while the 
student is enrolled or at exit. It is not surprising that most comprehensive institutions that collect 
basic skills data do so at the point of entry into the institution (96%). However, it is somewhat 
surprising that less than one-quarter (23%) of these institutions report that basic skills are assessed 
while the student is enrolled. More than half of comprehensive institutions that collect data on 
students’ higher-order cognitive skills, general education competencies, competencies in the major 
and vocational skills do so while the student is enrolled (ranging from 63% to 73% of institutions). 
Most comprehensive institutions also collect data on exiting students’ competencies in the major 
(72%) and vocational skills (67%). In fact, comprehensive institutions collect this exit information 
from their students more often than do all institutions in our study. 

At first glance, these data suggest that comprehensive institutions are assessing their 
students’ major competencies and vocational skills at more than one point in time. But in actuality, 
less than one-third (30%) of comprehensive institutions are collecting major competence data from 
their students both while they are enrolled and at exit, and less than one-quarter (24%) do so for 
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vocational/professional skills. The remaining institutions are collecting major competence and 
vocational skills data either while their students are still enrolled or at exit. 

Timing of Collecting Affective and Behavioral Data 

With the exception of academic intentions, most affective data are collected either during 
enrollment or at exit. In terms of academic intentions, it again is not surprising that students' 
intentions are collected at entry (86%). It seems apparent that these intentions are less often 
assessed again, even though students’ goals may change throughout their time at an institution. A 
large proportion of institutions report collecting data on students' personal growth, involvement 
and satisfaction with the institution during enrollment and at exit, but only one-third (35%) of 
institutions report collecting any of these types of data both while students are enrolled and at exit. 
There are no notable differences between the timing of comprehensive institutions’ data collection 
efforts and those of all institutions in our study. 

Student Assessment Instruments and Methods Used 

Institutions engaging in student assessment efforts must select the means by which 
assessment data will be collected. A traditional choice is to use assessment instruments, generally 
in the form of objective examinations or inventories administered in a pencil and paper or 
computerized format. More recently, there has been growing interest in alternative methods of 
assessing students. These non-traditional methods tend to be more qualitative or integrative in 
approach — requiring students to demonstrate higher-order cognitive skills such as application or 
synthesis — or may use sources of information other than students themselves. Further, as the 
student body entering postsecondary institutions becomes more diverse through the increased 
enrollment of part-time, older and minority students, some scholars have suggested the need to use 
special assessment methods to reflect the unique characteristics, learning styles, needs and life 
situations of various sub-populations of students. 

Student Assessment Instruments 

How are institutions collecting student assessment data? Survey respondents reported 
whether their institution uses instruments or tests to collect any of ten types of assessment 
information: nine for current students (basic college-readiness skills: higher-order skills; general 
education competencies; competence in major field of study; vocational or professional skills; 
personal growth and affective development; experiences and involvement with the institution; 
satisfaction with the institution; and academic intentions) and one for former students (alumni 
satisfaction and experiences). If an instrument was used, respondents specified its source — 
institutionally-developed, state-provided, or commercially available. Figures 13 and 14 present 
information on these eleven types of assessment information collected via traditional instruments or 
tests for comprehensive institutions and all institutions in our study. 

As Figures 13 and 14 show, the use of assessment instruments or tests varies with the type 
of assessment data being collected. Comprehensive institutions are most likely to use instruments 
or inventories to collect data on student and alumni satisfaction (97% and 96% respectively), basic 
college-readiness skills (95%), and major field competencies (91%) and least likely to use them to 
collect data on students’ vocational or professional skills (63%), higher-order cognitive skills 
(66%), and general education competencies (69%). 

When an instrument is used, it has most often been developed by the comprehensive 
institution itself (data concerning the source of instruments for each type of data are available in 
Appendix A of this report). This is especially true for instruments measuring aspects of student 
performance or experiences that are specific to the institution such as alumni satisfaction, student 
satisfaction, student effort and experiences, and competence in the major. Conversely, 
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comprehensive institutions make slightly higher use of commercially-available instruments for 
measuring basic college-readiness skills and higher-order cognitive skills. These measures are both 
more uniform across campus settings and more complex to develop. Comprehensive institutions 
make limited use of state-provided instruments, suggesting that such instruments are not a pan of 
state-level requirements for student assessment. 

Figure 13. Institutional Use of Student Assessment Instruments to Collect Cognitive Data 
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Comprehensive institutions do differ from all institutions in a few aspects of their use of 
assessment instruments and tests. Compared to all institutions, they make greater use of 
institutionally-developed instruments to collect data on students’ basic college-readiness skills and 
competencies in the major, and more often use commercially-available instruments to collect data 
on students’ major field competencies. This greater emphasis on major field competencies reflects 
the professional preparation focus of these institutions relative to other types. 
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Other Student Assessment Methods 



Institutions also reported the extent to which they use each of nine alternative methods of 
student assessment. Table 3 presents this information for comprehensive institutions and all 
institutions. Four of these options require more active participation of currently enrolled students 
and factored together to create a “student-centered” index of assessment methods that we used in 
regression analyses presented in section eight: observations of student performance, student 
portfolios or comprehensive projects, capstone courses, and student interviews or focus groups. 
None is used extensively by comprehensive institutions. Of these four methods, comprehensive 
institutions are most likely to use capstone courses (mean score of 2.4 1 ) and observations of 
student performance (mean score of 2.24) — doing so in some to most units or departments. 



Table 3. Mean Scores of Extent of Use of Other Student Assessment Methods 



Student Assessment Methods. 


Comprehensives (n=314) 


All Institutions (n=!393) 


Student-Centered 






Observations of student performance 


2.24 


2.26 


Student portfolios or comprehensive projects 


2.18 


2.10 


Student performance in capstone courses 


2.41 


2.15 


Student interviews or focus groups 


1.98 


1.84 


Extemallv-Oriented 


Alumni interviews or focus groups 


2.03 


1 .90 


Employer interviews or focus groups 


1.87 


1.87 


Other 


Transcript analysis 


2.13 


2.16 


External examination of students 


1.08 


2.02 


Surveys or interviews with withdrawing students 


2.35 


2.40 



I = not used: 2 = used in some units: 3 = used in most units: 4 = used in all units 



Two of these alternative measures rely on information from external groups or sources and 
factored together to create an indices of “external methods” of student assessment that we used in 
regression analyses in section eight: alumni interviews or focus groups, and employer interviews 
or focus groups. Comprehensive institutions are slightly more likely to interview or hold focus 
groups with alumni (mean score of 2.03) than with employers (1.87), but do not make extensive 
use of either method. 

Three additional alternative student assessment measures were listed in our survey: 
transcript analysis, using external examinations, and surveying or interviewing withdrawing 
students. Of these three, comprehensive institutions are most likely to survey/interview 
withdrawing students (2.35) followed by the use of transcript analysis (2.13). Of all the alternative 
methods considered here, comprehensive institutions are least likely to use external examinations 
such as licensure examinations as a method of collecting assessment data (1.08). It appears that 
comprehensive institutions consider former students to be more important referent group for 
assessment purposes than employers or professional/disciplinary groups. 

Compared to all institutions participating in our study, comprehensive institutions make 
greater use of capstone courses as an assessment method. However, they only use this method in 
some to most of their units. Conversely, they make much less use of external examination data. 
Other alternative assessment methods are not used to any greater or lesser extent by comprehensive 
institutions compared to all institutions. 
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Assessment Methods for Student Sub-Populations 

Do institutions use different assessment methods for specific sub-populations of their 
student body? Table 4 presents the percentage of comprehensive institutions and all institutions 
using different assessment methods for four student sub-populations: adult students, pan-time 
students, minority students, and distance education students. 



Table 4. Institutional Use of Different Assessment Methods for Different Student Sub- 
Populations 



Student Sub-Population 


9c Institutions Usiniz 


Different Assessment Methods 


Comprehensives (n=31 1 ) 


All Institutions (n=l 366) 


Adult students 


16 


10 


Part-time students 


5 


5 


Minority students 


3 


2 


Distance education students 


25 


22 



Most comprehensive institutions do not use different assessment methods for specific sub- 
populations of undergraduate students. While one-quarter (25%) report using different methods for 
distance education students, a decision that may be necessitated by the type of medium used for 
course delivery, very few use special methods for collecting assessment data from their adult, part- 
time or minority students. This profile is consistent across all types of postsecondary institutions. 

Student Assessment Studies and Reports 

Beyond collecting descriptive data on their students’ characteristics, performance or 
development, institutions must consider how to transform that data into useful information. 
Institutions that study the relationship between various aspects of students’ institutional 
experiences and their performance will be better able to make informed decisions concerning 
academic and student-related policies and practices. Another consideration in data analysis is the 
level of aggregation at which assessment information is analyzed and reported. 

Student Assessment Studies 

Are institutions studying how student performance is connected to experiences with the 
institution? In our survey, we asked if institutions conduct studies of the relationship between 
students’ performance and nine areas of students’ institutional experiences. Table 5 identifies these 
areas and presents data on institutional studies for both comprehensive institutions and all 
institutions in our study. 

Fully one-third of comprehensive institutions (35%) do not conduct any of these studies of 
students’ institutional experiences. When studies are conducted, they are most likely to examine the 
relationship of admissions policies (52%), residence arrangements (32%), financial aid and 
employment (31%), and extra-curricular involvement (30%) to students’ performance. In fact, 
comprehensive institutions conduct studies of the first two of these areas much more often than do 
all institutions. 

Nearly one-third (29%) of comprehensive institutions study the relationship of academic 
advising to student performance. However, they are much less likely to study the relationship 
between students’ performance and other aspects of students’ experiences that are more directly 
related to the teachtng/leaming environment such as interaction with faculty (13%), academic 
resources (14%) and exposure to various types of teaching methods (19%). Arguably, studies of 
these relationships may produce information that could have a greater impact on student 



performance but they are also more politically sensitive for institutions to conduct. Or it may be that 
studies of this nature are primarily conducted by faculty members within their own classrooms 
rather than by a central committee or institutional office: such studies would not have been reported 
in this institutional-level instrument. 



Table 5, Institutions Conducting Student Assessment Studies 



Studies of Relationship Between Student Performance 
and the Following Experiences 


c /c Institutions Conducting Studies 
Comprehensives (n=302) All Institutions tn=1264) 


Do not study any of these relationships 


35 


38 


Admission standards or policies 


52 


42 


Residence arrangements 


32 


21 


Student Financial aid and/or concurrent employment 


31 


30 


Extra-curricular activities 


30 


24 


Academic advising patterns 


29 


26 


Student course-taking patterns 


24 


26 


Exposure to different instructional or teaching methods 


19 


21 


Classroom, library and/or computing resources 


14 


17 


Patterns of student-faculty interaction 


13 


14 



Student Assessment Profiles or Reports 

Finally, institutions were asked to report the levels of aggregation at which they provide 
profiles or reports of student assessment information. Figure 15 shows the percentage of 
comprehensive institutions and all institutions providing student assessment reports at each of five 
levels of aggregation: institution wide; schools or colleges; academic programs or departments: 
special populations of students; course or groups of courses. Respondents indicated as many levels 
of aggregation as were applicable or could indicate that they provide no reports of student 
assessment information. 

Figure 15. Preparation of Student Performance Reports by Level of Aggregation 




institution school program subgroup course no reports 



Most comprehensive institutions do provide some type of reports of student assessment 
results (93%). The majority provide such reports on data for the institution as a whole (74%) and 



aggregated by academic programs or departments (76%). Half prepare reports at the level of 
schools and colleges (54%) and for specific sub-populations of students (51%). They are least 
likely to aggregate and report assessment information at the course level (27%). 

Compared to all institutions, comprehensive institutions are more likely to provide reports 
at the school or college level, and by academic programs or departments. This suggests that the 
interpretation of student assessment data may be more decentralized in these institutions, with 
schools and/or departments having the responsibility to decide what assessment results mean and 
what to do with them than is the case for all postsecondary institutions. 

Summary Observations 

In summary, comprehensive institutions tend to collect more extensive data on cognitive 
rather than affective or behavioral measures, and from current rather than former students. These 
findings are partly reflective of institutional mission. Unlike the tendency of associate of arts 
institutions to emphasize vocational preparation and liberal arts colleges to focus on students' 
cognitive and affective development, comprehensive institutions emphasize students’ acquisition of 
professional or disciplinary knowledge. However, this data collection profile may also be partly 
attributable to the ease or difficulty of collecting various types of data. Measures of academic 
progress and basic skills are easier for institutions to define, quantify and collect than more 
complex measures such as higher-order cognitive skills and affective development. Overall, the 
profile of data collection within comprehensive institutions closely resembles the profile reported 
by respondents from all types of postsecondary institutions. Compared to other types of 
institutions, comprehensive institutions generally fall in the mid-range of scores: some differences 
are evident but these are seldom large. 

Comprehensive institutions make great use of instruments or inventories to collect 
assessment data and most often develop these instruments themselves. Overall, they make limited 
use of non-traditional assessment methods such as portfolios and capstone courses. 

Compared to all institutions, comprehensive institutions have a moderately strong profile of 
analyzing and reporting student assessment data. However, studies are more likely to examine the 
influence of non-instructional factors such as admissions or financial aid policies on student 
performance than faculty-related factors. A greater proportion of comprehensive institutions, and 
all institutions, provide descriptive reports of student performance — most often aggregated at the 
institution-wide or department level. 



5. ORGANIZATIONAL AND ADMINISTRATIVE SUPPORT PATTERNS 



Focuses on the patterns of organizational and administrative support that 
institutions have developed to promote student assessment on their campuses, 
such as their mission and purpose, institution-wide activities, leadership 
support, and planning and coordinating mechanisms. Many comprehensive 
institutions have adopted a decentralized approach to assessment policies and 
planning — permitting academic units to develop their own assessment 
activities and involving faculty in planning and overseeing assessment 

activities. 

An important focus of our study was to examine the patterns of organizational and 
administrative support institutions have developed to promote student assessment on their 
campuses. The assessment literature suggests that institutions will be most likely to promote 
internal support for assessment if they engage in assessment for internal rather than external 
purposes, have visible and strong leadership support for assessment, include a broad range of 
internal participants, particularly faculty, in making assessment-related decisions and regularly 
evaluate their assessment programs. It is expected that organizational and administrative support 
will shape the assessment approach, assessment management policies and practices, and 
ultimately, the institutional uses and impacts of assessment information. 

The ISSA instrument included questions concerning the institution-wide assessment 
support strategy; administrative and governance activities used to promote student assessment; 
leadership and faculty support for assessment; planning and coordination for assessment; and 
evaluation of student assessment processes. Once again, we will present information concerning 
the pattern of organizational and administrative support for student assessment in comprehensive 
institutions and compare it to the pattern for all types of postsecondary institutions. 

Institution-Wide Assessment Support Strategy 



Mission Emphasis 

Assessment scholars and practitioners suggest that institutions' academic mission content 
may symbolize the importance with which student assessment is regarded and so may influence the 
degree of internal support for assessment. What do the mission statements of our responding 
institutions tell us about their values regarding student assessment? Institutions reported whether 
their mission statement explicitly: a) emphasizes excellence in undergraduate education, b) 
identifies intended student outcomes, c) refers to student assessment as an important priority, or d) 
does not mention any of these. Respondents could answer “yes” to more than one mission 
statement component. Figure 16 displays the percentage of comprehensive institutions and all 
institutions reporting each of these mission statement components. 

Despite their dual emphasis on graduate and undergraduate education, the majority of 
comprehensive colleges and universities report their mission statement explicitly emphasizes 
excellence in undergraduate education as an institutional priority (89%). However, just over half 
have mission statements that identify the educational outcomes intended for their students (53%) 
while less than one-quarter refer to student assessment as an important institutional activity within 
their mission statement (22%). Very few comprehensive institutions have none of these 
components in their mission statement (7%). Compared to all institutions, comprehensive 
institutions are more likely to emphasize undergraduate education, educational outcomes and 
student assessment in their mission statements. However, these differences are not large. 



Figure 16. Institutional Mission Emphasis 
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Purposes for Engaging in Student Assessment 

* 

Why are institutions engaging in student assessment? A second dimension of institution- 
wide assessment support strategy examined in this study is the purpose(s) underlying institutions' 
student assessment efforts. Scholars contend that whether assessment is primarily engaged in for 
internal or external purposes may influence the nature of an institution's assessment approach, 
degree of internal support, and assessment uses and impacts. Institutions rated the importance of 
six purposes for engaging in student assessment: preparing for an institutional accreditation self- 
study; meeting state reporting requirements; guiding internal resource allocation decisions; guiding 
undergraduate academic program improvement; improving the achievement of undergraduate 
students; and improving faculty instructional performance. The last four of these purposes were 
factored together to create an “internal purposes’’ index that we used in regression analyses in 
section eight. Figure 17 presents the mean scores for each student assessment purpose for 
comprehensive institutions and for all institutions. 



Figure 17. Purposes of Student Assessment 
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Comprehensive colleges and universities rate preparing for an institutional accreditation 
self-study (3.67), improving the achievement of undergraduate students (3.50), and guiding 




undergraduate academic program improvement (3.46) as very important reasons for engagin': in 
student assessment. These findings suggest regional accrediting associations play an important role 
in stimulating institutions' decisions to engage in student assessment. To a slightly lesser extent, 
comprehensive institutions intend their student assessment efforts to contribute to improvements in 
academic and student performance. 

Conversely, comprehensive institutions give the lowest importance ratings to guiding 
internal resource allocation decisions (2.62), meeting state reporting requirements (2.76) and 
improving faculty instructional performance (2.95) as student assessment purposes. All are rated 
as being of minor to moderate importance. It appears that student assessment efforts in 
comprehensive institutions are more often intended for improvement purposes than to address 
issues of external or internal accountability. The relatively low importance rating given to 
improving faculty instructional performance may stem from the moderate degree of faculty 
autonomy in these institutions. 



Compared to all institutions, comprehensive institutions accord slightly less importance to 
preparing for accreditation self-studies and meeting state requirements as student assessment 
purposes. This suggests these external constituencies may have slightly less influence over student 
assessment efforts in comprehensive institutions than other types of institutions. 

Institution-Wide Administrative and Governance Activities 

Institutions may develop a variety of administrative activities and governance structures to 
promote student assessment on an institution-wide basis. How' prevalent are these practices? The 
ISSA instrument asked whether or not institutions use any of the following administrative activities 
and governance structures: annual institution-wide initiatives on assessment; rewards or incentives 
for academic and student affairs administrators to use student assessment in their unit; incentives 
for academic units to use assessment information in evaluation and improvement efforts; student 
assessment workshops for academic and student affairs administrators; board of trustees committee 
that addresses student assessment; faculty governance committee that addresses student assessment 
issues; and student representation on student assessment committees. Figure 18 displays the 
percentage of comprehensive institutions and all institutions that have introduced these activities or 
structures in their institutions. 

Figure 18. Institution-Wide Administrative and Governance Activities Used to Promote 
Student Assessment 
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Of the various activities listed, comprehensive institutions are most likely to have a faculty 
governance committee that addresses student assessment issues (64%) and to provide workshops 
on student assessment for their academic and student affairs administrators (61%). A smaller 
proportion use annual institution-wide initiatives on assessment (41%), have student representation 
on student assessment committees (41%) and offer incentives for academic units to use assessment 
information in improvement efforts (31%). Comprehensive institutions are much less likely to have 
a board of trustees committee focused on student assessment (17%) and least likely to offer 
rewards or incentives to individual administrators who promote student assessment (7%). 

Relative to all institutions, comprehensive institutions are slightly more likely to involve 
faculty and students in assessment decision-making. Their use of the remaining administrative and 
governance activities basically parallels that of all institutions. 



Leadership and Faculty Support for Student Assessment 

The degree to which student assessment efforts are supported internally is likely an 
important determinant of the extent and impact of an institution's student assessment activities. 
How supportive of student assessment are various internal constituencies? We asked survey 
respondents to rate the degree to which six internal groups support undergraduate student 
assessment activities: board of trustees; chief executive officer; academic affairs administrators; 
student affairs administrators; faculty governance; and students. Figure 19 presents the mean 
responses to this question for comprehensive institutions and all institutions in our study. 



Figure 19. Internal Constituent Support for Student Assessment 
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Scale: 1 = very unsupporlive; 2 = somewhat unsupporlive; 3 = neutral, unknown; 4 = somewhat supportive; 
5 = very supportive 



Comprehensive institutions have generally high levels of support for student assessment 
with most constituents reported as being at least somewhat supportive. Academic affairs 
administrators are perceived as the most supportive constituency followed by the chief executive 
officer and student affairs administrators. Boards of trustees and faculty governance bodies are 
viewed as slightly less supportive in relation to senior administrators but are still rated as 
supportive. Students are seen as being least supportive of student assessment with most 
respondents reporting their level of support as neutral or unknown. This finding may warrant 
further attention given scholars’ concerns about the need to actively promote student participation 



in assessment activities. In all cases, comprehensive institutions report their constituent groups as 
being equally supportive of assessment as those in all institutions in our study. 

Planning and Coordinating Student Assessment 

A central issue in discussions of leadership and governance for student assessment 
concerns the degree to which responsibility for planning and coordinating student assessment 
activities is centralized or decentralized within institutions. Centralized approaches that focus such 
responsibility on senior administrators or central offices may signal that assessment is a valued 
institutional activity but decentralized approaches involving an array of internal participants may do 
more to promote broader internal support for assessment. How do institutions plan and coordinate 
their assessment efforts? We asked institutional respondents about six aspects of their assessment 
planning and coordination process: the nature of an institutional plan or policy for student 
assessment; membership on an institution-wide planning group for assessment; executive 
responsibility for the assessment planning process; approval authority for assessment plans; 
operational responsibility for student assessment; and reporting patterns. 

Institutional Plan or Policy for Student Assessment 



We asked survey respondents which of seven types of institutional plans or policies for 
student assessment best described what exists at their institution: 1) formal comprehensive 
centralization — a formally adopted plan or policy specifying undergraduate student assessment 
activities for all academic programs or units; 2) formal limited centralization — a formally adopted 
plan or policy for undergraduate student assessment in some academic programs or units; 3) formal 
decentralization — a formally adopted institutional plan or policy requiringlill academic units or 
programs to develop their own undergraduate student assessment plan; 4fformal guidance — a 
formally adopted institutional plan or policy identifying institution-wide activities to be conducted 
by a central committee or office; 5) informal encouragement — no institutional plan or policy but 
academic units or programs are encouraged to develop their own undergraduate student assessment 
activities; 6) emergent — institution is currently developing a plan or policy for undergraduate 
student assessment; and 7) none — institution does not have an undergraduate student assessment 
plan or policy. Respondents could select more than one type of plan or policy. Figure 20 presents 
the percentage of comprehensive institutions and all institutions in our study using each type of 
assessment plan or policy. 



Figure 20. Types of Institutional Plan or Policy for Student Assessment 
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Virtually all comprehensive institutions have some type of institutional plan or policy for 
undergraduate student assessment. Almost half ( 47 %) report having a plan specifying assessment 
activities of all academic units or program and two-fifths (407c) stipulate institution-wide 
assessment activities that are conducted by a central committee or position, but more than half 
(577c) require all academic units or programs to develop their own undergraduate student 
assessment plan. So it appears that while requirements to engage in student assessment are 
centrally stipulated, responsibility for determining the specific form these activities will take is 
often left to the discretion of academic units or programs. Comprehensive institutions are more 
likely than all institutions in our study to have adopted a decentralized assessment plan or policy. 
They make limited use of the other types of policies and plans considered in this question. 

Institution-Wide Assessment Planning Group Membership 



Who is involved in student assessment planning? We asked respondents whether they had 
an institution-wide group responsible for ongoing planning and policy setting for undergraduate 

• assessment. Fully 73% of comprehensive institutions and 70% of all institutions report having 
such a group. Of those institutions with institution-wide planning groups, we asked which of the 
following internal constituents serves on the group: chief executive officer, academic affairs 
administrator or staff, student affairs administrator or staff, institutional research administrator or 
staff, academic review and evaluation administrator or staff, student assessment administrator or 
staff, faculty, and students. Respondents checked as many constituents as were applicable. Figure 

® 21 presents the percentage of comprehensive institutions and all institutions that responded that 

each constituent is a member of their assessment planning group. 

When comprehensive colleges and universities have institution-wide planning committees 
for student assessment, these committees are predominantly comprised of faculty (93%) and 
academic affairs administrators (87%). Institutional research personnel (59%), student affairs 

• administrators (47%) and students (387o) are represented to a lesser degree. It is least likely for the 
chief executive officer (7%) and for personnel with specific responsibilities for academic 
review/evaluation ( 22 %) or student assessment (327o) to be included. The latter two positions are 
uncommon within comprehensive institutions. This membership profile suggests that student 
assessment planning is primarily positioned as an academic affairs responsibility. This holds true 
for both comprehensive institutions and for all institutions in our study. 

• 

Figure 21. Membership on Institution-Wide Student Assessment Planning Group* 




*Only institutions with an institution-wide planning group for student assessment responded to this question. 



O 

ERIC 



34 



42 





Executive Responsibility for Assessment Planning Process 

Another measure of the governance process used for student assessment concerns where 
formal leadership for assessment planning is vested in an institution. We asked institutions which 
of the following positions has executive responsibility for or chairs the institution-wide group for 
student assessment planning: academic affairs administrator: student affairs administrator: 
institutional research officer; academic review’ and evaluation officer: student assessment officer: or 
faculty member. Figure 22 presents this information for comprehensive institutions and all 
institutions. 



Figure 22. Executive Responsibility for Student Assessment Planning Group* 
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*Only institutions with an institution-wide planning group for student assessment responded to this question. 



Executive responsibility for student assessment planning in comprehensive institutions is 
most often placed in the hands of an academic administrator (54%), followed by a faculty member 
(38%). It is less common for student affairs administrators or for personnel from institutional 
research, academic review and evaluation or student assessment to hold this position. Compared to 
all institutions, comprehensive institutions are slightly more likely to vest executive responsibility 
for planning in a faculty member (38% of comprehensive institutions versus 3 1 % of all 
institutions). 

Approval Authority for Student Assessment Plan 

As a final aspect of governance for assessment planning, we asked institutions to identify 
who, among ten possible positions or groups, approves changes in their institutional plan or policy 
for student assessment: board of trustees; chief executive officer; chief academic affairs officer; 
chief student affairs officer; institutional research officer; academic review and evaluation officer; 
student assessment officer; student government; academic senate or other faculty committee; and' 
faculty union. All respondents could answer this question regardless of whether or not they had an 
institution-wide committee for assessment planning. Respondents indicated as many positions or 
groups as were applicable. Less than 10% of comprehensive institutions report that academic 
review and evaluation, student assessment, student government or faculty union personnel have 
approval authority for assessment planning. Figure 23 presents the results for the remaining six 
positions for comprehensive institutions and all institutions. 



Figure 23. Approval Authority for Student Assessment Plan or Policies 
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Changes to student assessment plans or policies have to be approved by multiple sources. 
However, comprehensive institutions most often give approval authority to the chief academic 
affairs officer (80%). In terms of frequency, faculty members are the internal constituency 
mentioned next (48%) followed by the chief executive officer (37%). There is minimal 
involvement of other administrative or research/evaluation positions, or the board of trustees in this 
aspect of assessment governance. Compared to all institutions, comprehensive institutions are 
more likely to involve the academic senate or other faculty committees and less likely to include the 
chief executive officer in approving changes to the institutional assessment plan or policy. 

Operational Responsibility for Day-to-Day Student Assessment Activities 

Beyond the planning phase of student assessment, institutions must decide where to place 
responsibility for overseeing their day-to-day undergraduate student assessment activities such as 
instrument development, data collection, analysis and reporting. Where have institutions placed 
operational responsibility for student assessment? We asked institutions which of six positions or 
offices has responsibility for day-to-day student assessment activities: academic affairs 
administrator; student affairs administrator; institutional research officer; academic review and 
evaluation officer; student assessment officer; and faculty member(s). Institutions checked as many 
positions/offices as applied. Figure 24 presents the results for comprehensive institutions and all 
institutions. 

Comprehensive institutions most often hold institutional research personnel responsible for 
their day-to-day student assessment activities (47%), a finding that makes sense given the 
measurement and statistical proficiency required for executing these tasks. However, this 
responsibility is also frequently given to academic affairs administrators (44%) and faculty 
members (39%). Conversely, comprehensive institutions are less likely to have a student affairs 
administrator, academic review/evaluation officer or student assessment officer in charge of their 
day-to-day assessment activities. Comprehensive institutions are slightly more likely to call upon 
faculty members to manage the daily operations of student assessment than all institutions in our 
study. 

Reporting Relationship of Individual with Operating Responsibility for Student Assessment 

Finally, institutions were asked to whom the individual with day-to-day operating 
responsibility for student assessment reports. Survey respondents selected from the following five 





offices: chief executive officer; chief academic officer; chief student affairs officer; institutional 
research officer; and academic review and evaluation officer. Figure 25 presents the responses 
from comprehensive institutions and all institutions in our study. 

Figure 25. Reporting Relationship of Individual with Operating Responsibility for Student 
Assessment Activities 
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In the majority of comprehensive institutions (71%), the individual with operational 
responsibility for student assessment reports to the chief academic officer. In a much smaller 
proportion of institutions, this individual reports to the chief executive officer ( 19%). Compared to 
all institutions in our study, comprehensive institutions more often mention reporting to the chief 
academic officer and less often mention reporting to the chief executive officer. Other offices 
considered in this question were almost never cited by comprehensive institutions and all 
institutions alike. 



Evaluating the Student Assessment Process 

The student assessment literature insists upon the importance of institutions regularly 
evaluating their assessment processes. Consequently, our survey inquired whether institutions had 
• evaluated their student assessment plan or process and if so, what elements of their plan or process 

had been reviewed. 
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Status of Evaluation of Student Assessment Plan or Process 



Institutions were asked whether they have conducted a formal evaluation, an informal 
evaluation, are currently developing plans for an evaluation, or are not planning to evaluate their 
assessment process. The results for comprehensive institutions and all institutions are depicted in 
Figure 26. 



Figure 26. Status of Student Assessment Evaluation 



O) 

c 



o 

Q_ 

CD 

CC 

c/) 

c 

o 

3 



c/> 

C 



vP 




formal informal planning an no 

evaluation evaluation evaluation evaluation 



Only half of comprehensive institutions have evaluated their student assessment process, 
more often informally (30%) than formally (19%). About one-third are developing evaluation plans 
(31%). Comparatively few (19%) are neither evaluating nor planning to evaluate their assessment 
process. There are few differences in the status of evaluation among comprehensive institutions 
and all institutions in our study. The relatively low incidence of actual evaluation and high 
incidence of evaluation planning is reflective of the newness of assessment activities among 
postsecondary institutions. 

Elements of Student Assessment Evaluation 

Those institutions that had formally or informally evaluated their student assessment 
process were asked which of eight elements were reviewed as part of the evaluation: student 
assessment plans and policies; structure and responsibility for student assessment; achievement of 
intended objectives for student assessment; reliability and validity of assessment instruments and 
methods; quality of data analysis; use of assessment information in institutional decision-making; 
problems encountered while conducting assessment; and the costs and benefits of student 
assessment. Results of this question for comprehensive institutions and all institutions are 
presented in Figure 27. 

When evaluating their student assessment process, comprehensive institutions most often 
focus on administrative or managerial elements such as the assessment plan itself (79%), 
achievement of intended objectives (74%), problems encountered while conducting assessment 
(74%), and structure and responsibility for assessment (68%). There is somewhat less examination 
of the use of assessment information in institutional decision making (60%) or of the technical 
aspects of assessment such as instrument reliability and validity (52%) and data analysis quality 
(54%). Institutions are least likely to evaluate the comparative costs and benefits of student 
assessment (23%). Comprehensive institutions differ little from all institutions in their evaluation 
approaches. 



Figure 27. Elements of Student Assessment Evaluation 
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Summary Observations 

The preceding sections have examined various aspects of comprehensive institutions' 
organizational and administrative support for student assessment. The overall pattern reveals a 
student assessment support strategy that is more internally than externally focused in its 
orientation, and a leadership and governance approach that is quite decentralized in nature. 
Evaluation efforts appear to be still emerging. 

The mission statements of comprehensive colleges and universities prioritize undergraduate 
education although few make explicit mention of assessing student performance. In terms of the 
overall purpose or function of their student assessment efforts, survey responses from 
comprehensive institutions suggest they are more often intended to address internal than external 
needs. With the exception of preparing for an institutional accreditation self-study, comprehensive 
institutions identify internal improvements — in student achievement, academic programs, and to a 
lesser extent, faculty instruction — as more important purposes for engaging in student assessment 
than purposes related to external or internal accountability. 



Considering their use of administrative and governance activities, comprehensive 
institutions rely more on professional development opportunities and governance structures that 
engage internal constituents in decision making as means of promoting assessment than on 
administrative incentives or rewards. Internal leadership support for student assessment is 
generally thought to be high; senior administrators are viewed as the internal constituency most 
supportive of student assessment, followed by faculty and trustees. 

Overall, comprehensive institutions have adopted a decentralized approach to assessment 
plans and policies. This is reflected both in the form of institutional policy for student assessment 
that typically exists and in their patterns of participation in assessment planning. Planning and 
coordinating responsibility for student assessment is chiefly situated in academic affairs rather than 
in chief executive, institutional research or student affairs offices. Faculty are well represented in 
all aspects of the student assessment planning process — from sitting on an institution-wide 
planning group, to holding executive responsibility for this group, to approving changes in the 
assessment plan or policy. Oversight of day-to-day student assessment activities is most often 
assigned to institutional research personnel and faculty members, with academic affairs personnel 
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also frequently given this responsibility. The individual with operational responsibility most often 
reports to the chief academic officer. 



Evaluation of the student assessment plan and process is not yet a well-developed practice 
among comprehensive institutions. Fully half of comprehensive institutions have not vet evaluated 
their assessment process although many are planning to do so. However, most elements of a good 
evaluation are considered by comprehensive institutions that have either formally or informally 
evaluated their student assessment process. 

Compared to all institutions, comprehensive institutions are more likely to have a formal 
decentralized institutional policy for student assessment — requiring some form of student 
assessment activity of all academic units but permitting units to develop their own assessment plans 
and activities. Student assessment is more often positioned as an academic affairs responsibility 
than as a student affairs or administrative function. Comprehensive institutions are more likely to 
involve faculty and less likely to involve student affairs personnel in student assessment planning 
and managing or overseeing day-to-day assessment activities. The chief academic officer is more 
often designated and chief executive officer is less often designated as the reporting line for 
changes to the assessment plan or day-to-day assessment operations. 
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6. ASSESSMENT MANAGEMENT PRACTICES AND POLICIES 



Reports on the existence of specific institutional practices aiul policies 
designed to promote student assessment management. Comprehensive 
institutions prefer unobtrusive practices and policies , such as distributing 
assessment results and providing access to student performance information, 
to more obtrusive practices and policies such as using student assessment 
information in making either budget allocation or faculty evaluation and 
. reward decisions. 

A fourth domain in our conceptual framework is that of assessment management practices 
and policies that provide mechanisms for managing the student assessment process and direct the 
ways in which student assessment information is used throughout the institution. Assessment 
management practices and policies are suggested in the literature as powerful means through which 
institutions can support student assessment and encourage the use of collected assessment 
information. Conceptual dimensions of assessment management practices and policies such as their 
comprehensiveness, consistency, and the extent to which they are employed within an institution 
are expected to influence internal support for student assessment and the likelihood of achieving 
institutional impacts from assessment information. 

Our survey asked about specific institutional practices and policies promoting student 
assessment management. These items were factor analyzed to create nine comprehensive indices. 

In this chapter we will present results of these assessment management practice and policy indices, 
rather than of individual items. 

Practices for Managing Student Assessment 

What kinds of practices do institutions develop to manage and promote their student 
assessment process? We asked respondents about the existence of a variety of specific institutional 
practices used to manage the student assessment process in four areas: academic resource 
allocation, student information systems, internal access to student assessment information, and 
distribution of assessment reports and studies. Scores for institutions for all specific practices in 
these four sections are presented in Appendix A. We used factor analysis to reduce these responses 
to indices of management practices in these four areas. Detailed information on these indices is 
provided in Appendix B. Briefly, the four indices are: 

1 . Academic Budget Decisions: whether the institution’s academic budget process compares 
academic units on student performance indicators and allocates resources competitively among 
them; or rewards them for improvement based on student performance indicators. 

2. Computer Support: whether the institution has a computerized student information system 
which includes student performance indicators, tracks students from application through 
graduation; and is integrated with faculty, curricular and financial databases. 

3. Access to Student Assessment Information: whether assessment information for individual 
students is available to institutional research, assessment or evaluation professionals; senior 
academic administrators; department chairs or academic program administrators; student affairs 
professionals; and faculty advisors. 

4. Distribution of Assessment Reports: whether assessment reports are regularly distributed to 
students, faculty, academic administrators, student affairs professionals, employers, and the 
general public. 
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Figure 28 presents mean scores for these four indices. Each was originally measured on a 
different scale. For the purposes of comparison, we have translated each index to a scale of zero to 
Five. Therefore, we can determine which of these practices is more prevalent in comprehensive 
institutions and all institutions in our study. Scores for specific items comprising each index are 
taken from Appendix A. 



Figure 28. Extent of Institutional Use of Assessment Management Practices 
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Scale: 0 = institution uses no practices in index; 5 = institution uses all practices in index. 



Of these four indices of assessment management practices, comprehensive institutions are 
£ most likely to provide access to assessment information to a number of internal constituents (mean 

index score of 3.34). For example, one of the items comprising this index is whether this 
information is accessible to institutional researchers: 77 % of comprehensive institutions in our 
study report that it is. High proportions of comprehensive institutions (63% to 76%) report that 
student assessment information is available to a variety of academic personnel. There is one 
exception to this pattern of internal access: only half (49%) of comprehensive institutions grant 
0 such information access to student affairs professionals, a proportion that is lower than that 

reported by all institutions in our study. 

The next most prevalent student assessment management practice is distributing student 
assessment reports to six internal and external constituents (mean index score of 1 . 95 ). 
Comprehensive institutions are most likely to regularly distribute assessment reports to academic 
9 administrators (87%), faculty ( 68 %) and, to a lesser extent, student affairs professionals (52%). 

Very few regularly distribute such reports to students (17%), employers (4%) or the general public 
(7%). Compared to all institutions, comprehensive institutions are less likely to distribute 
assessment reports to student affairs personnel; there are no large differences in other report 
distribution practices. 

0 Comparatively speaking, comprehensive institutions have limited information system 

capabilities for student assessment (mean index score of 1.25). One of the items in this index is 
whether the institution has a student information system that tracks students from application 
through to graduation: less than half (43%) of comprehensive institutions in our study have such a 
system. Fewer (24%) have a computerized information system which includes student 
performance indicators and fewer still ( 8 %) have a student assessment information system that is 
9 integrated with other institutional databases. Comprehensive institutions do not report much 

different information system capabilities than all institutions in our study. 
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Using student assessment information in the budget process to compare and reward 
academic units is almost a non-existent practice (mean index score of .25). Very few- 
comprehensive institutions use student performance indicators as the basis for competitively 
allocating resources among academic units ( 2 %) or rewarding academic units for performance 
improvement (5%). Comprehensive institutions do not differ appreciably from all institutions in 
their use of these resource allocation practices. 

Policies Supporting and Promoting the Use of Student Assessment 

While the previous section focused on practices used to manage student assessment, this 
section focuses on the policies used to both support student assessment and guide how the 
resulting data are incorporated into other institutional processes. We asked survey respondents 
about the extent to which they have developed a variety of specific institutional policies to support 
and promote student assessment in five areas: student involvement in assessment; professional 
development for faculty and academic affairs administrators; training for student affairs personnel; 
faculty evaluation and rewards; and academic planning and review processes. Scores for 
institutions for all specific policies are presented in Appendix A. Factor analysis produced five 
indices of student assessment policies. Detailed information on these indices is provided in 
Appendix B. Briefly, these five indices are: 

1 . Student Involvement: extent to which the institution requires students to participate in student 
assessment activities, provides students with information on the puipose and uses of student 
assessment, and provides individual feedback regarding student performance results. 

2. Professional Development: extent to which the institution provides funds for faculty to attend 
assessment conferences, faculty workshops or consultative services on student assessment, 
assistance to faculty to improve their use of student assessment, and workshops/seminars for 
academic administrators on assessment. 

3. Student Affairs Training: extent to which the institution requires student affairs staff to receive 
training on assessment and provides student assessment workshops for student affairs 
administrators. 

4. Faculty Evaluation: extent to which the institution considers evidence of student performance in 
faculty evaluation for promotion; incorporates evidence of student performance into faculty 
evaluation for salary and merit; considers faculty scholarship on assessment in promotion, 
tenure or salary reviews; considers faculty participation in assessment in promotion, tenure or 
salary reviews; and recognizes faculty for effective use of assessment. 

5. Academic Planning and Review: extent to which the institution incorporates student 
performance data into academic department or undergraduate program planning or review; 
general education or core curriculum review; course-level review and development; and review 
and planning for student academic support services. 

These indices summarize the extent to which these policies exist at institutions based on a 
five point scale: 1 = not done at all; 2 = done in a few departments; 3 = done in some departments; 

4 = done in many departments; 5 = done in most departments. Figure 29 presents the mean scores 
for these five indices for comprehensive institutions and all institutions in our study. Mean scores 
for individual policies comprising each index are taken from Appendix A. 

Of the five indices of student assessment policies presented in Figure 29, comprehensive 
institutions are most likely to incorporate student assessment data into academic planning and 
review processes (mean index score of 2.72). They make fairly extensive use of student 
assessment data in planning and review processes for academic departments or undergraduate 



Figure 29. Extent of Institutional Use of Assessment Management Policies 
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1-not done at all; 2-done in a few depts; 3=done in some depts; 4=done in many depts; 5=done in most depts. 



programs (mean item score of 3.78), general education or core curriculum (mean item score of 
3.42), and courses (mean item score of 3.24) — doing so in some to many departments. They are 
slightly less likely to use this information in processes for reviewing and planning student 
academic support services (mean item score of 2.92). 

Comprehensive institutions also make fairly extensive use of policies encouraging student 
involvement in assessment activities (mean index score of 2.56). Institutional scores for specific 
policies comprising this index reveal that, on average, some to many departments require students 
to participate in assessment activities (mean item score of 3.66) and provide students with 
information regarding the purposes and uses of student assessment (mean item score of 3.42). 
Some departments provide students with individual feedback regarding their assessment results 
(mean item score ot 2.97) but only a few departments provide incentives as a means of 
encouraging student involvement in assessment activities (mean item score of 1.91). 

Comprehensive institutions are less likely to offer professional development on student 
assessment to their faculty and academic administrators (mean index score of 1.81). Some 
departments provide funds for faculty to attend assessment conferences (mean item score of 2.85), 
and offer workshops or seminars on using student assessment information to their faculty (mean 
item score of 2.83) and academic administrators (mean item score of 2.50). They are less likely to 
encourage faculty use of assessment information by providing assistance such as paid leaves or 
course reduction (mean item score of 1.96). They also make limited provision of professional 
development on student assessment for student affairs staff or administrators (mean item score of 
1.78). Compared to all institutions, comprehensive institutions make slightly less extensive use of 
professional development policies to support their student assessment efforts. 

Many departments within comprehensive institutions report that they encourage their 
faculty to assess student learning in classes (mean item score of 3.81). However, respondents 
report that a few departments or less at their institution consider evidence of student performance in 
decisions concerning faculty evaluation for promotion (mean item score of 1.94) or annual salary 
and merit increases (mean item score of 1.60). They are slightly more likely to consider faculty 
scholarship on assessment (mean item score of 2.27) and participation in assessment (mean item 
score of 2.04) in evaluation decisions. A few departments publicly recognize faculty members who 
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have made effective use of student assessment (mean item score of 1.58). But on the whole, like 
all institutions in our survey, comprehensive institutions do not link faculty evaluation and reward 
policies to student assessment (mean index score of 1.31). 

Summary Observations 

In the previous section we concluded that patterns of organizational and administrative 
support for student assessment are decentralized within comprehensive institutions, more often 
directed toward addressing internal than external purposes, and that responsibility and coordination 
for student assessment is typically positioned within academic affairs. Findings regarding the use 
of assessment management practices and policies are consistent with these patterns. 

Comprehensive institutions permit a variety of academic personnel access to assessment 
information on individual students. Conceivably, providing this information access should assist 
faculty and academic administrators in helping students through advising or intervening as 
appropriate. Such access is less often extended to student affairs personnel. Assessment 
information distribution is most often intended for internal rather than external purposes, and more 
often for academic administrators than for other internal constituencies. While a sizable proportion 
of comprehensive institutions have the benefit of computer technology for managing their student 
information, comparatively fewer have merged assessment data within their information systems. 
This may constrain the ability of comprehensive institutions to systematically analyze student 
assessment data and to examine how student performance is related to aspects of students’ 
educational experiences. There is very little use of student assessment information in determining 
resource allocations among academic units. 

Comprehensive institutions make fairly extensive use of student assessment information in 
academic planning and review processes but less often incorporate this information in planning for 
academic support sendees. Some departments have policies intended to encourage student 
involvement in assessment activities. While many departments report that they encourage their 
faculty members to assess student learning in their classes, few provide professional development 
support for student assessment and even fewer include assessment-related criteria within their 
faculty evaluation and reward policies. 



7. USES AND IMPACTS OF STUDENT ASSESSMENT 



Explores two important questions regarding how institutions use student 
assessment information in decision-making and the impact it has on 
institutions. Comprehensive institutions are not using student assessment 
information to any great degree in making academic decisions and report that 
student assessment has had little impact on either their internal processes or 
their external relationships. 

As noted in our introduction, effective student assessment processes contribute to 
improvements in institutional and student performance. Consequently, two important questions in 
our research are: 1) how have institutions used student assessment information and 2) what impact 
has student assessment information had on institutions? 



From the literature we identified and focused on two critical dimensions: the use of student 
assessment information in academic decision making, and the internal and external impacts on the 
institution that have resulted from student assessment. Our survey included ten academic decisions 
and fourteen institutional impacts. Survey respondents used a four-point scale to indicate the extent 
to which student assessment information had influenced these academic decisions in the institution 
( 1 = no action or influence unknown; 2 = action taken, data not influential; 3 = action taken, data 
somewhat influential; and 4 = action taken, data very influential). Survey respondents also used a 
four-point scale to indicate the impact that student assessment information has had on the internal 
and external items indicating institutional performance (1 = not monitored, do not know; 2 = 
monitored, negative impact; 3 = monitored, no known impact; 4 = monitored, positive impact). 
Institutions’ scores for each use and impact item are presented in Appendix A. Details of the 
indices are presented in Appendix B. Factor analysis clustered these individual items into five 
indices of assessment management policies, two reflecting assessment information uses in 
academic decisions and three reflecting impacts of student assessment on the institution. Briefly, 
the five indices are: 



1 . Educational Decisions: the extent to which student assessment information has been used to 
revise undergraduate academic mission or goals; design or reorganize academic programs or 
majors; design or reorganize student affairs units; allocate resources to academic units; modify 
student assessment plans, policies, or processes; modify general education curriculum; modify 
student out-of-class learning experiences; create or modify distance learning initiatives; modify 
teaching methods; and modify student academic support services. 



9 



Faculty Decisions: the extent to which student assessment information has been used to decide 
faculty promotion and tenure; and faculty salary increases or rewards. 



3. Faculty Impacts: whether student assessment information has stimulated campus discussions of 
undergraduate education; contributed to faculty satisfaction, contributed to faculty interest in 
teaching; and led to changes in teaching methods used. 

• 4. Student Impacts: whether student assessment information has contributed to student 

satisfaction; affected student retention or graduation rates; affected student grade performance; 
and affected student achievement on external examinations. 



5. External Impacts: whether student assessment information has affected student application or 
acceptance rates; allocation of state funding; evaluation from regional accreditation agency; 

• private fund-raising results; success on grant applications; communications with external 

constituents; and institutional reputation or image. 
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Figure 30 presents the means for these Five indices for comprehensive institutions and all 
institutions. Mean scores for individual item comprising each index are taken from Appendix A. 

Figure 30. Institutional Uses and Impacts of Student Assessment Information 
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*Use scale: l=no action or influence unknown, 2=action taken, data not influential; 3=action taken, data somewhat 
influential; 4=action taken, data very influential. 

**Impact scale: l=not monitored, do not know; 2=monitored, negative impact; 3=monitored. no known impact: 
4=monitored, positive impact. 



Uses in Academic Decisions 

As Figure 30 shows, student assessment information has had limited influence on 
educational decisions (mean index score of 1.44) within comprehensive institutions and even less 
on faculty-related decisions (mean index score of 1 .30). A comparison of institutions' mean scores 
for specific items comprising each of these indices provides more detail about the uses made of 
assessment information. A description of these indices and their associated items is available in 
Appendix B. Mean scores for items are presented in Appendix A. 

Concerning individual items within the educational decisions index, comprehensive 
institutions have most often used student assessment information to design or reorganize academic 
programs or majors (mean item score of 2.67) and to modify student assessment plans or 
processes (mean item score of 2.60), student academic support services (mean item score of 2.56) 
and general education curriculum (mean item score of 2.55). These mean scores indicate 
assessment information has been somewhat influential in these decisions. They are least likely to 
have used assessment information to create or modify distance learning initiatives (mean item score 
of 1.70) and to allocate resources to academic units (mean item score of 1.79). 



In contrast, more than two-thirds of respondents from comprehensive institutions are 
9 unaware of the influence of assessment information on decisions concerning faculty promotion and 

tenure (67%), and faculty salary increases or rewards (70%), the individual items comprising the 
faculty decisions index. For the most part, comprehensive institutions report that they are not at all 
likely to use student assessment information in making. decisions on either faculty promotion and 
tenure (mean item score of 1.45) or faculty salary increases or rewards (mean item score of 1.45). 
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Compared to all institutions in our study, comprehensive institutions make greater use of 
assessment information in educational decisions and less use in faculty decisions. But overall, 
assessment information appears to have limited influence on institutional decision making. 

Institutional Impacts 

In terms of student assessment impacts on the institution, comprehensive institutions are 
slightly more likely to report student assessment information has affected indicators of faculty 
(mean index score of 1.58) and student (mean index score of 1.58) performance than external 
indicators of institutional performance (mean index score of 1 . 1 8). Again, a comparison of 
institutions’ mean scores for specific items comprising these three indices provides more detail 
about the impact of student assessment information. A description of these indices and their 
associated items is available in Appendix B. Mean scores for items are presented in Appendix A. 

Within the faculty impacts index, comprehensive institutions are most likely to attribute 
changes in faculty teaching methods (mean item score of 2.46) and more frequent discussions of 
undergraduate education (mean item score of 2.41) to student assessment information. They have 
more often documented positive impacts from assessment information on student retention or 
graduation rates (mean item score of 2.15) and satisfaction (mean item score of 2.04) than on 
student grades (mean item score of 1 .80) or performance on external examinations (mean item 
score of 1.94). 

Institutions are least likely to report that student assessment has had an impact on external 
measures of institutional performance. With one exception, mean scores of individual items 
comprising this index indicate that most comprehensive institutions have not monitored external 
impacts of their student assessment efforts (mean scores range from 1.43 to 1.99). The exception 
to this pattern concerns regional accreditation evaluations: institutions are more likely to report a 
positive impact from student assessment information on this indicator (mean item score of 2.66) 
than any other individual measure of external performance considered in the survey. 

Overall, as the mean scores presented in Figure 30 clearly show, most institutions 
participating in our study, comprehensive institutions and all institutions alike, have not monitored 
the impact of student assessment on internal and external indicators of institutional performance. 
Comprehensive institutions do not differ appreciably from all institutions in our study in their 
reports of student assessment impacts. 

Summary Observations 

To date, comprehensive institutions have made limited use of student assessment 
information for institutional decisions. Many respondents are unaware of whether assessment 
information has been influential or not in shaping institutional decisions. When used, assessment 
information is more likely to inform educational decisions, particularly concerning academic and 
assessment planning, than decisions regarding faculty evaluation and rewards. 

Similarly, very few institutions have monitored the impact of student assessment 
information on faculty and student performance and even fewer have done so for external 
indicators of institutional performance. When monitoring has been undertaken, comprehensive 
institutions most often report assessment information has stimulated discussions of undergraduate 
education and led to changes in teaching methods used. These Findings indicate that student 
assessment has not yet become entrenched in comprehensive institutions. 



8. KEY RELATIONSHIPS 



Institutional factors, such as conducting assessment for internal improvement 
purposes and providing a number of administrative and governance activities 
to promote student assessment, have more influence on the type and extent o f 
student assessment data collected by comprehensive institutions than do state 
or regional accreditation factors. Institutions are more likely to report using 
student assessment information in academic decisions if they make extensive 
use of a variety ? of assessment management practices and policies to promote 
their assessment efforts, conduct studies of collected assessment data and use 
institution-wide support strategies. External forces are comparatively weaker 
influences on the use of assessment information. Similarly, institutional 
choices and activities are stronger determinants of achieving positive impacts 
from assessment information than external influences. Specifically, 
comprehensive institutions that collect and analyze extensive assessment data, 
have centralized policies requiring assessment activities, and use assessment 
management practices and policies to support assessment efforts report that 
student assessment has had a greater impact on faculty and student 
performance and external relationships. 

In the previous sections we have examined external influences, institutional approaches, 
organizational and administrative support, management practices and policies, and institutional 
uses and impacts related to student assessment. But even more important is understanding how 
these domains influence the likelihood that student assessment will make a difference in 
institutional performance. That is, which external influences, institutional approaches to 
assessment, organizational and administrative support patterns, and assessment management 
practices and policies are most likely to promote the use and positive impacts of student assessment 
information? 

In this section, we examine three key relationships. First, we discuss how external 
influences — the state and regional accrediting agencies — influence an institution's approach to 
student assessment. Then we examine how external influences and institutional characteristics, 
approaches, support patterns, and management practices and polices affect an institution's use of 
student assessment data. Finally, we examine how these domains lead to positive institutional 
impacts from student assessment. The analyses in this section use the indices discussed in previous 
sections. For detailed information on the indices, please refer to Appendix B. 

External and Internal Influences on Student Assessment Approaches 

Are external forces more influential than internal forces in determining how a 
comprehensive institutions approach student assessment? We used three multiple regression 
models to compare the influences of state characteristics related to student assessment, regional 
accreditation membership, and institution-wide support patterns for student assessment on the 
extent of an institution’s use of three approaches to student assessment: cognitive assessment (the 
extent to which data are collected on measures of students’ cognitive performance); affective 
assessment (the extent to which data are collected on measures of students’ affective development 
or performance); and post-college assessment (the extent to which data are collected on measures 
of students’ post-enrollment performance). In these three models, the predictor variables stem from 
three domains: state assessment approach, accrediting region, and institution-wide support. The 
predictor variables are shown in Table 6 and defined in detail in Appendix B. These analysis were 
restricted to public comprehensive institutions. 
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Predictors of Cognitive Assessment 



Which of the variables has the most influence on whether a comprehensive institution will 
assess students' cognitive abilities? The first two columns of Table 6 show how institutional, state 
and regional accrediting influences are related to the extent to which these institutions collect data 
on students’ cognitive performance. This model predicted a moderate proportion of the variance in 
institutions’ collection of cognitive data, accounting for 25% of the variance. 



Table 6. The Influence of Institution-Wide Support, State Assessment Approach, and 
Accrediting Region on the Extent of Institutional Approach to Student Assessment for Public 
Comprehensive Institutions 

Extent of Institutional Approach 



Cognitive Assessment Affective Assessment 
Beta AR : BetaAR - 



Post-College 

Assessment 

Beta AR : 



R 2 . 25 ** , 07 ** . 06 ** 

Institution-Wide Support 

Mission emphasis 
Administrative & governance 
activities 

Administrator & faculty 
support 

Conduct for internal 
improvement 
Conduct for state 
Conduct for accreditation 
State Assessment Approach 

Authority structure 
Form of state assessment 
initiative 

Common indicators/outcomes 
Accrediting Region 



Middle States 
North Central 


29** 


.05 










New England 
Northwest*** 


-.16* 


.02 










Southern 

Western 


-.20** 


.03 


92** 


.05 


-.16* 


.03 



*p < .05; **p < .01 

***Since “accrediting region” was a categorical variable. Northwestern accrediting region was left out of this 
regression because its effect on cognitive competencies, based on ANOVA, was closest to the mean. 



.15* 


.02 










.15* 


.02 


.15 


.02 






2] ** * 


.1 1 






.17* 


.03 



Institution-wide patterns of support are the strongest and a positive source of influence. In 
particular, institutions that view internal institutional improvement as an important purpose for 
conducting student assessment are likely to collect the most extensive cognitive data. The number 
of administrative and governance activities sponsored by the institution to promote student 
assessment and the degree of administrator and faculty support for student assessment are also 
important predictors of cognitive assessment. 
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Regional accrediting membership is also related to institutions' cognitive data collection 
efforts but the significant influences from this domain are negative. That is. institutions in the 
Middle States, Western and New England regional accrediting regions are less likely to assess 
students’ cognitive abilities than institutions belonging to the other accrediting regions. This may 
reflect variations in the length of time different accrediting regions have included student 
assessment criteria within their reporting requirements for institutions as well as differences in the 
specific emphases of those criteria. No state variables emerged as significant predictors of 
cognitive assessment. 

Predictors of Affective Assessment 

The middle two columns of Table 6 show' the relationship of variables in the institutional, 
state and regional accrediting domains to the extent to which comprehensive institutions collect data 
on students’ affective development. The variables in this model do not work well as predictors of 
affective assessment: 

the total model explains only 7% of the difference in affective assessment efforts among 
comprehensive institutions. Only two variables are statistically significant predictors in this model. 
Comprehensive institutions that belong to the Southern accrediting region are likely to collect more 
extensive affective data than those that belong to other accrediting regions. The degree of 
administrator and faculty support for student assessment is also positively related to collecting 
affective data. No state variables serve as significant influences on affective assessment. 

The poor ability of this model to predict institutions' affective assessment efforts is largely 
attributable to our earlier finding that very few comprehensive institutions collect this type of data. 
There is simply not much assessment behavior here to predict! This finding also suggests that 
variables not included in this model may account for institutions’ decisions to collect affective data 
from their students. 

Predictors of Post-College Assessment 

The final two columns of Table 6 show the relationship of variables in the institutional, 
state and regional accrediting domains with the extent to which comprehensive institutions collect 
data from former students (post-college assessment). Only 6 % of the variation in this type of data 
collection is accounted for by this model. Again, only two variables operate as significant 
influences. Conducting assessment for internal improvement is predictive of the extent to which 
comprehensive institutions collect post-college data. Institutions in the Western accrediting region 
are less likely to collect assessment data from formerly-enrolled students. 

As was the case with affective data, this model does not work well at predicting 
institutions’ data collection for former students. This suggests that other variables not included in 
this model are more important influences on this dimension of student assessment approach. 

Institution-wide support strategies and accreditation region membership are the primary 
influences in all three of these student assessment approach models. Apparently, the drive for state- 
level accountability has not exceeded the influence of institutional accreditation, and internal 
strategies are the driving force of all three approaches to student assessment. 

Influences on Using Assessment Information in Academic Decisions 

How do comprehensive institutions effectively promote and support the use of student 
assessment information in academic decision making? Within the domain of academic decision 
making, we created two indices reflecting the use of student assessment information: educational 
decision making and faculty decision making (these indices are described in detail in Appendix B 
and section seven). We used multiple regression to examine the influence of external forces. 



institutional size, institutional approach to student assessment, institution-wide support, and 
assessment management practices and policies on using student assessment information in 
educational and faculty decisions. The predictor variables in this model are defined in detail in 
Appendix B. Table 7 presents the results of these two regression models for comprehensive 
institutions. 

Influences on the Use of Assessment Information in Educational Decisions 

The educational decisions model works very well for comprehensive institutions, 
explaining almost half (49%) of the variance in the influence of student assessment data on 
educational decisions. Significant predictor variables come from three institutional domains: 
approach to assessment, institution-wide support, and assessment management practices and 
policies. 

Of these, the domain of assessment management practices and policies contributes the 
strongest and most influences. In particular, the extent to which comprehensive institutions have 
policies encouraging student involvement is strongly related to using assessment data in 
educational decisions (accounts for 18% of variance) followed by the existence of policies 
involving student affairs staff in professional development for student assessment (accounts for 
5% of variance). While comprehensive institutions report quite extensive use of student 
involvement policies, most make comparatively little use of policies for their student affairs 
personnel. Including assessment-related criteria in faculty evaluation and reward processes is also 
positively related to using assessment data in educational decisions (accounts for 2% of variance). 
This policy and the use of assessment information in educational decisions are both related to a 
strong internal commitment to student assessment. Three other assessment practices and policies 
are also positively related to educational decision uses although each contributes little on its own: 
the breadth of internal access to assessment information, distribution of assessment summary 
reports, and providing professional development on student assessment for faculty and academic 
administrators. 

One variable from the assessment approach domain functions as a very strong predictor. 
The number of studies conducted of the relationship of student performance and institutional 
experiences is positively related to using assessment data to make educational decisions (accounts 
for 1 1% of variance). This is a logical relationship; conducting such studies should provide 
institutions with relevant data to consider in decisions concerning academic planning, instructional 
practices and academic support services — all decisions reflected within the educational decisions 
index. Two other variables are also significantly related to educational decision uses although each 
contributes little to the variance in this assessment use. Collecting cognitive data from students is a 
positive predictor of educational decision uses while the number of instruments used to collect 
assessment data is a negative predictor of using assessment information to make educational 
decisions. Instruments and inventories generally consist of questions that are relevant to students 
enrolled in a variety of institutions. Information collected from these instruments, while useful for 
measuring student performance, may not provide much specific information for administrative 
decision making at individual institutions. 

Finally, three institution-wide support variables are predictive of using assessment 
information in educational decisions: conducting student assessment for internal improvement 
purposes (explains 5% of variance), evaluating the student assessment process (explains 3% of 
variance), and emphasizing undergraduate education and student assessment in the academic 
mission (explains 2% of variance). These three variables are indicative of a strong institutional 
commitment to student assessment. 

No external influence variables are significant predictors of educational decisions, 
suggesting that this use of assessment information is primarily determined by internal influences. 



Table 7 . The Influence of External Influences, Institutional Context, Institutional Approach, 



Institution-Wide Support, and Management 


Practices 


and Policies 


on Using Student 




Assessment Information in Educational and 


Faculty 


Decisions for 


Comprehensive Institution.* 




Educational Decisions Beta 


Faculty Decisions Beta AR~ 






AR : 






Adjusted R : 


49 * * 


,20** 




External Influences 










Middle States accrediting region 
North Central accrediting region 
New England accrediting region 
Southern accrediting region 
Western accrediting region 
State initiative for student assessment 






.13* 


.01 


State approach to student assessment 






] 7 ** 


.02 


Accrediting influence 
Institutional Context 






1 2 * 


.01 


Control (1 = public, 2 = private) 
Enrollment 






-.13* 


.02 


Institutional Approach to Student Assessment 










Cognitive assessment 
Affective assessment 


.11* 


.01 






Post-college assessment 
Number of instruments 


-.13* 


.01 






Student-centered methods 
External methods 
Total assessment studies 


21 ** 


.1 1 






Institution-Wide Support for Student Assessment 










Mission emphasis 


12** 


.02 






Conduct for internal improvement 
Conduct for state 
Conduct for accreditation 


21** 


.05 






Administrative & governance activities 
Administrator & faculty support 
Formal centralized policy 
Institution-wide planning group 
Conducted evaluation of assessment process 


19** 


.03 


.14* 


.01 


Assessment Management Policies and Practices 










Academic budget decisions 
Computer support 
Access to information 


.09* 


.01 


.18** 


.04 


Distribution of reports 


.09* 


.01 






Student involvement 


.11* 


.18 






Professional development 


. 10 * 


.01 


.16** 


.02 


Student affairs training 


.15** 


.€5 






Faculty evaluation 1 


.09* 


.02 


n/inc 




Academic planning and review 2 


n/inc 




1 9 ** 


.04 



*p < . 05 ; **/> < .01 

'The factor “faculty evaluation" was not entered into the regression model predicting use of student assessment information 
in faculty decisions as many of the items comprising these two factors were similar. 

2 The factor “academic planning and review" was not entered into the regression model predicting use of student assessment 
information in educational decisions as many of the items comprising these two factors were similar. 
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Nor were there significant differences in the likelihood of using assessment data for educational 
decisions by institutional control or size. 

Influences on the Use of Assessment Information in Faculty Decisions 

The variables used in this regression model have less influence on comprehensive 
institutions’ use of assessment data for making faculty-related decisions. In total, they account for 
only 20% of the variance in this assessment use. This result is not surprising, given that most 
comprehensive institutions do not use student assessment information to make faculty decisions. 

As we observed in the model for educational decisions, assessment management practices 
and policies are the strongest predictors of comprehensive institutions using assessment data to 
make faculty-related decisions. Institutions that incorporate assessment data in their academic 
planning and review processes (explains 4 % of variance) and that use assessment data as a basis 
for making resource allocations to academic units. (explains 4 % of variance) are more likely to use 
assessment data in decisions about faculty tenure, promotion, salary increases and other rewards. 
This suggests a general pattern of integrating assessment data with institutional decision making. 
Providing professional development on student assessment for faculty and academic administrators 
is also a positive predictor of using assessment data in faculty decisions (explains 2% of variance). 

External influences play a role in this use of assessment data although the influence is 
weaker than that from assessment management practices and policies. Institutions that report 
having state-mandated student performance indicators and outcomes are more likely to use 
assessment data for faculty decisions (explains 2% of variance). Institutions that view regional 
accreditation requirements as an important influence on their assessment activities and that belong 
to the Southern accrediting region also make greater use of assessment data in faculty decisions. 

Finally, it appears that private comprehensive institutions are less likely than public to use 
student assessment data to make faculty-related decisions. 

Influences of Student Assessment Information on Internal and External 

Institutional Performance 

How does the use of student assessment information affect various internal and external 
institutional performance dimensions? In our survey, comprehensive institutions reported whether, 
and the extent to which, student assessment has affected various aspects of faculty and student 
performance and relationships with their external environment (these indices are described in 
Appendix B and section seven). Using factor analysis, we created two indices of internal impacts 
from assessment: faculty impacts and student impacts. All of the items in the external impact 
section of the survey factored into one “external impact” index. We used multiple regression to 
examine the relationship of external forces, institutional approach to student assessment, 
institution-wide support patterns, and assessment management practices and policies to institutional 
impacts of student assessment information. We ran separate regression models for each of the three 
impacts. The predictor variables in these models are defined in detail in Appendix B. Table 8 
presents the results of the three regression models for comprehensive institutions. 

Predictors of Faculty Impacts 

What variables predict whether a comprehensive institution’s student assessment efforts 
will have an impact on its faculty members’ attitudes and activities? The first two columns of Table 
8 show statistically significant predictors of faculty impacts. According to the results of our 
regression model, the domains of institutional approach to student assessment, institution-wide 
support strategies for student assessment, and assessment management practices and policies are 
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Table 8. The Influence of External Influences, Institutional Context, Institutional Approach, 



Institution-Wide Support, and Management 


Practices and 


Policies on Faculty. 


, Student, and 


External Impacts for Comprehensive Institutions 








Faculty Impacts 


Student Impacts 


External Impacts 




Beta AR : 


Beta AR : 


Beta AR : 


Adjusted R 2 


^25 * * 


.22** 


2 ^ * 


External Influences 








Middle States accrediting region 
North Central accrediting region 
New England accrediting region 
Southern accrediting region 
Western accrediting region 
State initiative for student assessment 
State approach to student assessment 
Accrediting influence 




.16** .02 




Institutional Context 








Control ( 1 = public, 2 = private) 
Enrollment 






.13* .02 


Institutional Approach to Student Assessment 








Cognitive assessment 
Affective assessment 
Post-college assessment 






.12* .01 


Number of instruments 
Student-centered methods 
External methods 






.13* .03 


''Total assessment studies 

Institution-Wide Support for Student Assessment 


24** |2 


.27** .13 




Mission emphasis 

Conduct for internal improvement 






.11* .02 


Conduct for state 
Conduct for accreditation 
Administrative & governance activities 
Administrator & faculty support 
Formal centralized policy 


.14** .06 


.13* .02 


.16** .04 


Institution-wide planning group 
Conducted evaluation of assessment process 


16** .03 






Assessment Management Policies and Practices 








Budget decisions 
Computer support 
Access to information 
Distribution of reports 
Student involvement 


15** .03 


.14** .02 


.20** .09 


Professional development 
Student affairs involvement 
Faculty evaluation 
Academic planning and review 


.14* .02 


.16** .05 





*p < .05; **/? < .01 
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all important influences on faculty impacts from assessment. Together, these predictors explained 
25% of the variance in faculty impacts of student assessment. 

One dimension of the student assessment approach undertaken plays a key role in achieving 
faculty impacts. Comprehensive institutions that conduct studies of the relationship between 
students’ performance to their interactions with the institution are likely to report that their 
assessment activities have impacted faculty (explains 12% of variance). This finding reinforces the 
view of assessment scholars that institutions must do more than simply collect assessment data if 
student assessment is to inform changes in institutions' teaching and learning processes. They need 
to commit efforts to analyze their assessment results in view of students' institutional experiences. 

Comprehensive institutions that have a formal centralized policy for student assessment — 
that is, a formally adopted plan or policy that specifies undergraduate student assessment activities 
for all academic programs or units — are more likely to report faculty impacts from assessment 
(accounts for 6% of variance), as are institutions that have either formally or informally evaluated 
their student assessment process (accounts for 3% of variance). These institution-wide support 
practices may signal the degree to which institutional leaders support and value undergraduate 
student assessment. 

The assessment management practices and policies used by the institution are also 
important determinants of whether their assessment activities will affect its faculty. Comprehensive 
institutions that distribute summary reports to a wide array of internal and external personnel and 
that offer professional development on student assessment to faculty and academic administrators 
are more likely to report that their assessment activities have impacted faculty (respectively account 
for 3% and 2% of variance). 

Conversely, external influences from state and regional accrediting bodies and institutional 
characteristics of size and control are not important predictors of faculty impacts from student 
assessment. 

Predictors of Student Impacts 

What variables predict whether a comprehensive institution's student assessment efforts 
will have an impact on the performance of its students? The middle two columns of Table 8 reveal 
that institutional choices and activities from the domains of assessment approach, institution-wide 
support and assessment management practices and policies influence the likelihood of achieving 
student impacts from assessment. This model explains 22% of the variance in student impacts of 
student assessment. 

Once again, the number of studies conducted of the relationship between students’ 
performance and their institutional experiences is the strongest predictor of a comprehensive 
institution’s assessment efforts leading to improved student performance (accounts for 13% of 
variance). In the first place, institutions that conduct such studies are likely to be more aware of 
changes in student performance. Secondly, these institutions are in a more informed position to 
modify aspects of students’ experiences within the institution in ways that will promote student 
learning and development. 

Having an institutional policy that stipulates undergraduate student assessment activities to 
be carried out within all academic programs or units is also a strong predictor of achieving positive 
student impacts from assessment (explains 2% of variance). As noted in the discussion of faculty 
impacts above, this centralized approach to assessment may be indicative of a strong internal 
commitment to undergraduate student assessment. 
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Two assessment management practices and policies used by a comprehensive institution 
also influence whether its student assessment activities will impact student performance. 

Institutions that use student assessment data in their academic planning and review processes and 
that use student performance indicators to allocate resources among academic units are likely to 
report that their assessment activities have contributed to improvements in student performance 
(account for 5% and 2% of variance respectively). These findings support a positive relationship 
between using student assessment data for institutional decision making and achieving positive 
benefits from student assessment. 

Only one external variable is significantly related to whether or not comprehensive 
institutions attribute positive student impacts to their assessment efforts. Institutions that are 
members of the New England accreditation region are more likely to do so than those that belong to 
other accrediting regions. Beyond this, external influences and broad institutional characteristics 
appear to play a minor role in promoting student impacts from assessment. 

Predictors of External Impacts 

Lastly, what variables predict whether a comprehensive institution's student assessment 
efforts will have an impact on its relationships with the external environment? The final two 
columns of Table 8 show the statistically significant predictors of external impacts. Overall, this 
model explains 23% of the variance in achieving external impacts from student assessment. 

The strongest predictor overall comes from the domain of assessment management 
practices and policies. Comprehensive institutions that distribute summary reports of assessment 
data to a wide array of constituencies are more likely to report that engaging in student assessment 
has a positive external impact (accounts for 9% of variance ). This breadth of information 
distribution demonstrates an institution’s commitment to informing internal and external personnel 
about how its student body is performing; positive reports are likely to enhance an institution’s 
external image and reputation. 

From the institution-wide support domain, two variables emerge as strong predictors of 
external impacts from assessment. Comprehensive institutions that consider meeting state reporting 
requirements to be an important purpose underlying their student assessment efforts are more likely 
to report positive external impacts, among them, changes in their allocation of state funding 
(accounts for 4% of variance). Institutions for which internal improvement is an important reason 
for engaging in student assessment are also more likely to attribute positive external impacts to their 
student assessment efforts (accounts for 2% of variance). Presumably, these institutions will be 
able to report improvements in institutional and student performance to their external 
constituencies. 

Within the domain of institutional approach to student assessment, comprehensive 
institutions that collect data from their former students (accounts for 1% of variance) and those that 
use a great number of instruments to collect student data (accounts for 3% of variance) are more 
likely to report positive external impacts from student assessment. The relationship between 
collecting post-college assessment data and achieving positive external impacts is logical; these 
institutions will have more data available concerning the performance of their formerly-enrolled 
students within the external community and so will be in a better position to monitor external 
impacts. As for the connection between instrument use and external impacts, it may be that data 
from assessment instruments are more easily quantifiable and thus easier to report to external 
constituencies than data that are collected through more qualitative assessment methods. 

One institutional context variable is significantly predictive of achieving external impacts 
from assessment. Larger comprehensive institutions report more positive external impacts from 
student assessment than smaller institutions (accounts for 2% of variance). Institutions supporting 



a larger enrollment base may be more likely to use their assessment results as a means of marketing 
their institution to prospective students and funding sources. 

Summary Observations 

By comparing how internal and external forces influence the content of comprehensive 
institutions' student assessment approaches, it is clear that internal influences are much stronger. 
While different accrediting regions spur varying levels of engagement in collecting student 
assessment data, state policies and structures do not influence the type and extent of assessment 
data collected by comprehensive institutions. Two internal strategies are particularly important in 
determining the extent of student data collected: conducting assessment for internal improvement 
purposes; and the degree of administrator and faculty support for student assessment. 

If comprehensive institutions are to benefit from engaging in student assessment, they 
should not only be collecting student data but also using these data to make academic decisions. 
Institutions that use student assessment data for academic planning, instructional development and 
other educational decisions tend to commit to their assessment process — evidenced by the 
strategies of engaging in student assessment for internal purposes, building assessment into their 
academic mission, not only collecting but also conducting relational analyses of their assessment 
data, and evaluating the assessment process itself. These institutions also make more extensive use 
of assessment management practices and policies to support assessment efforts; most notably, 
policies that encourage student involvement in assessment and provide professional development 
for student affairs personnel. External factors play some role in institutions' use of assessment data 
for faculty-related decisions, but assessment management practices and policies are even stronger 
influences. Linking assessment data to budget decisions and to academic planning and review 
processes is likely to coexist with using assessment data to decide faculty salary, promotion and 
tenure. 



If engaging in student assessment is truly making a difference in comprehensive 
institutions, they should report that it is having an impact on their internal performance and external 
relations. Internal choices and efforts are much more important predictors of achieving impacts 
from assessment information than influences from state and regional accreditation agents. 
Comprehensive institutions reporting positive faculty and student impacts have gone beyond 
merely collecting assessment data to analyzing and using the data they collect. They conduct 
studies of the relationship between student performance and various aspects of their institutional 
experiences, widely distribute reports of assessment information, and use assessment data for 
academic planning and internal resource allocations. They demonstrate leadership support for 
student assessment by enacting centralized policies requiring specified assessment activities of all 
academic units and evaluating the student assessment process. They offer professional 
development opportunities to faculty and academic administrators to promote their involvement in 
student assessment efforts. 

External influences do play some role in the likelihood of comprehensive institutions 
documenting external impacts from assessment. Institutions conducting student assessment to 
fulfill state reporting requirements are more likely to achieve positive external impacts from their 
assessment activities. Institutional size is positively related to external impacts. But in the main, it 
is internal efforts that increase the chances of improving external relationships through student 
assessment. Widely distributing reports of assessment information, using comprehensive 
instruments to collect assessment data, collecting data from former students, and viewing internal 
improvement as an important reason for engaging in student assessment are all strong predictors of 
reaping positive external benefits from student assessment. 



9. EXECUTIVE SUMMARY: RESEARCH RESULTS AND RECOMMENDATIONS 



Recommendations for comprehensive institutions based on the results of our 
research are outlined in this section. This study and monograph highlight the 
importance of viewing student assessment from a critical institutional 
perspective and underscore three important realities: l) for student assessment 
to enhance student, faculty, academic, and institutional performance, it has to 
be viewed as an institutional process, not just a series of student assessment 
activities; 2) a great deal of organizational, administrative, and academic 
activity is or can be invested in initiating, managing, and using student 
assessment; and 3) a systematic look at those institutional activities can 
enhance an institution’s ability to plan for and use student assessment 
effectively. 

Student Assessment: A Critical Institutional Perspective 

The intent of our study and this monograph is to highlight the importance of viewing 
student assessment from a critical institutional perspective and to underscore three important^ 
realities: 1) for student assessment to enhance student, faculty, academic, and institutional 
performance, it has to be viewed as more than just a series of student assessment activities: 2) a 
great deal of organizational, administrative, and academic activity is or can be invested in initiating, 
managing, and using student assessment; and 3) a systematic look at those institutional activities 
can enhance an institution’s ability to use student assessment effectively. 

The data reported in this survey provide a national profile of what comprehensive 
institutions are currently doing to support and promote the use of student assessment. This 
evidence provides insight into the activities institutions are doing extensively, those which merit 
greater attention, and those which make a difference in improving institutional performance. We 
begin with a summary of our research findings and recommendations concerning specific 
assessment activities conducted within comprehensive institutions. Then we discuss how 
comprehensive institutions can use the Institutional Support for Student Assessment (ISSA) 
inventory and the Framework for Institutional Support for Student Assessment (Figure 1 of this 
monograph) as a guide for examining their student assessment process and functions. We conclude 
by suggesting that the results of the national survey used in conjunction with institutional self- 
examination can serve as the basis for redesigning or planning a student assessment process which 
can enhance an institution’s academic performance. 

Student Assessment in Comprehensive Institutions: Results and Recommendations from 

the National Survey 

Institutions that have committed resources to assessing the development and performance 
of their undergraduate students assessment activities should be able to use the student assessment 
data they collect in organizational decision-making and to document impacts from their assessment 
activities. Comprehensive institutions that are engaging in undergraduate student assessment, but 
do not feel that they are profiting from the process as much as they could be, may want to adopt the 
strategies of institutions that have reported using and being positively impacted by assessment 
information. Our research has demonstrated that most comprehensive institutions have made only 
limited use of student assessment data and given little attention to monitoring impacts from 
assessment. However, they are subject to external demands for assessment and have adopted a 
wide variety of student assessment approach measures, institution-wide activities supporting 
student assessment, and assessment management practices and policies. Our research shows that 
external influences do play some role in encouraging institutions to conduct and use student 



assessment, but there is much that institutions themselves can do to initiate, support and benefit 
from their undergraduate student assessment efforts. Our research identified strategies from the 
domains of student assessment approach, institution-wide support for student assessment, and 
assessment management practices and policies that are associated with comprehensive institutions 
reaping greater institutional uses and impacts from undergraduate student assessment. Greater uses 
and impacts mean that not only the institution as a whole benefits, but that faculty and students 
benefit as well. In the following sections, we highlight the student assessment activities that 
comprehensive institutions are currently engaging in to a great extent, the activities that they may 
want to augment, and the activities that are critical to enhancing the use of student assessment data 
in academic decisions leading to positive institutional impacts. 

Student Assessment Uses and Impacts 

If institutions are benefiting from their student assessment process, they should report that 
they are using the resulting assessment data in academic decisions and that the use of these data has 
a positive impact on both internal processes and performance and the institution's relationship with 
external constituents. In this section, we describe the extent to which comprehensive institutions 
are using collected student assessment data and the extent to which these data have had an impact 
on the institution. Suggestions for increasing institutional uses and impacts of student assessment 
are outlined in the following sections. 

Uses of Student Assessment Information . Most comprehensive institutions are not using 
student assessment data to make decisions concerning academic practices and policies or faculty. 
Our study did not attempt to discern why institutions are not making more extensive use of these 
data. Perhaps institutional decision-makers do not have sufficient data to inform decisions 
concerning institutional policies and practices. This may be particularly so if assessment data 
collection focuses on student inputs (for example, basic college-readiness skills or academic 
intentions of entering students) or outputs (for example, graduation or employment rates of exiting 
students) without attempting to measure changes in student performance or development, or 
without examining the relationship between student performance or development and facets of their 
experiences within the institution (for example, course-taking patterns, financial aid, advising 
policies). Assessment data use may also be limited if institutions have not created a formal 
mechanism for incorporating this data in decision-making processes. Comprehensive institutions 
that are collecting student assessment data but are not using these data in institutional decision 
making should examine why this is the case. For example, is the type of data being collected not 
useful for informing decisions? Are there concerns about data quality? Is assessment data not easily 
accessible or not widely distributed to decision makers? Raising questions of this nature will assist 
institutions in understanding how they may increase the use of assessment information in decision 
making. 



Impacts of Student Assessment Information . Similarly, comprehensive institutions report 
very minimal impacts from their student assessment data. We found that most institutions are 
simply not monitoring whether assessment information has had an impact. Of all institutional 
performance indicators considered in our survey, comprehensive institutions are most likely to 
attribute favorable evaluations from their regional accreditation agency, changes in the teaching 
methods used by their faculty, and increased discussions of undergraduate education to student 
assessment information. Some institutions have documented assessment-related increases in 
student retention and graduation rates, and student satisfaction. Comprehensive institutions have 
least often monitored the impact of assessment on student grades or on other dimensions of their 
relationships with external constituents such as state or private sector funding, student application 
and acceptance rates, and institutional reputation. Monitoring impacts is important if an institution 
is to evaluate the effectiveness of its student assessment efforts. As a first step, the institution must 
select the indicators of institutional and student performance it wants to monitor as well as the 
evidence it will use to measure these indicators. Which specific aspects of performance will be 



monitored, and the type of evidence used, will vary according to the institution's academic mission 
and its goals for student assessment. 

Regardless of the specific uses and impacts an institution may hope to achieve through 
student assessment, the remainder of this report suggests institutional practices, policies and 
strategies that may promote institutional uses and impacts of student assessment data. 

External Influences 

External influences, such as state and accreditation requirements, can be strong motivators 
for institutions to engage in or increase their student assessment efforts. In addition, a variety of 
postsecondary organizations and external funding sources offer support for institutions' 
assessment programs. This section describes the nature of these influences on comprehensive 
institutions and provides suggestions for deriving the most benefit from these external 
relationships. 

State Influences . Half of the comprehensive institutions that responded to our survey are 
under the purview of a state mandate for student assessment. Two-thirds of these institutions 
report that they worked together with state officials to develop the details of the state-level 
assessment plan. It follows that one-third of institutions were not involved in this development. 
Where such opportunities exist, institutions should try to participate in such planning at the state 
level. State-level assessment mandates that are jointly developed by institutional and state officials 
have a greater potential to be fair, practical, and ultimately beneficial to institutions. Characteristics 
of state-level initiatives (for example, whether the state assessment initiative is in the form of a 
policy or statute, or whether the state mandates specific indicators for institutions to report) have 
little direct influence on institutions’ data collection efforts or on the likelihood that they will use 
and be impacted by assessment information. However, our research shows that the importance 
institutions accord to meeting state reporting requirements as a purpose for engaging in student 
assessment is not significantly related to how extensively institutions collect assessment data, how 
much they use that data for decision-making, or how much they achieve positive impacts from 
using assessment data. Said differently, institutions that conduct student assessment solely to 
satisfy state requirements are not likely to reap internal benefits from assessment. Comprehensive 
institutions that must commit resources to student assessment because of state requirements are 
advised to also use these efforts for internal improvement purposes. 

Accreditation Influences . The majority of comprehensive institutions have completed a 
regional accreditation review requiring undergraduate student assessment. In fact, for most 
comprehensive institutions, accreditation requirements were either an important reason to initiate 
student assessment efforts or a stimulus to increase their involvement in assessing students. Some 
regional accreditation associations have more influence on institutions’ student assessment 
activities than others. For example, comprehensive institutions in the Middle States, Western and 
New England accrediting regions collect less cognitive assessment data than institutions belonging 
to other accrediting regions. Institutions in the Southern accrediting region are more likely to use 
assessment data in faculty decisions and to report positive impacts on student performance 
stemming from their assessment efforts. To some extent, these differences reflect differences in the 
approach regional accreditors have taken toward student assessment. Institutions should be 
cognizant of the influence of their accrediting regions and must, of course, respond to their 
requirements for assessment. However, comprehensive institutions should also bear in mind that if 
responding to accreditation requirements is the only major purpose for engaging in student 
assessment activities, these activities will not be likely to have an impact on how extensively 
institutions collect assessment data, how much they use that data for decision-making, or how 
much they achieve positive impacts from using assessment data. Accreditation requirements 
notwithstanding, comprehensive institutions that want to maximize assessment information uses 
and impacts are advised to make certain that the improvement of institutional performance (for 



example, improving academic programs and student achievement) is an important purpose 
underlying their assessment efforts. 

Other External Influences . In addition to state and regional accreditation requirements, there 
are other external resources comprehensive institutions should consider when engaging in student 
assessment activities. Many comprehensive institutions have made use of conferences and 
publications on student assessment provided by professional associations and regional 
accreditation associations. Although a variety of external agencies and programs offer grants to 
improve or support institutions’ assessment practices, the majority of comprehensive institutions 
have not received such grants. Comprehensive institutions interested in augmenting their 
assessment efforts may want to explore these opportunities for external support. 

Institutional Approach to Student Assessment. 

Institutional approach-to student assessment refers to institutional decisions regarding the 
collection and analysis of student assessment information. Dimensions along w'hich assessment 
approaches can be differentiated include: the type and extent of student assessment data collected; 
the methods used to collect assessment data; and the analyses conducted and reported for collected 
data. This section summarizes comprehensive institutions’ approaches to student assessment and 
offers recommendations for adopting assessment approaches that will lead to uses and impacts of 
student assessment information. 

Type and Extent . Most comprehensive institutions emphasize the collection of data on 
certain types of cognitive, affective and behavioral data from their students such as basic college- 
readiness skills, competencies in the major, general education competencies, academic progress, 
academic intentions, satisfaction, and their post-enrollment vocational and educational outcomes. 
These data are most often collected at one point in time during students' enrollment and provide 
static measures of students’ basic abilities, satisfaction and movement through the educational 
process that may be sufficient to satisfy accountability requirements. However, they provide 
institutions with less information concerning how their students are learning and changing while in 
college. Currently comprehensive institutions collect less extensive information on their students’ 
higher-order cognitive competencies, vocational and professional competencies, personal growth 
and affective development, involvement with the institution, and post-enrollment civic and social 
roles. Collecting these types of data, particularly at more than one point in time during students' 
involvement with the institution, would provide comprehensive institutions with more substantive 
data concerning students’ cognitive and affective growth. In addition, our research shows that the 
more assessment data institutions collect, the more likely they are to use and be positively affected 
by assessment information. If they are to achieve maximum benefit from assessment efforts, 
comprehensive institutions should review the type and extent of assessment data being collected 
and, where needed, broaden and deepen these collection efforts. 

Assessment Methods . How is assessment data currently being collected? Our survey 
results show comprehensive institutions tend to use tests and other written instruments to collect 
assessment data. These tests and inventories are often developed by institutions themselves, which 
may enhance their acceptance and use within the institution. Some units or departments make use 
of less traditional assessment methods such as capstone courses, observations of student 
performance, portfolios of student work, and surveys or interviews with withdrawing students. 
Institutions may wish to encourage even greater use of these alternative methods of collecting 
student assessment data. These alternative methods allow the assessment of higher-order skills 
such as application of facts, synthesis, and evaluation. They also tend to promote faculty 
involvement in assessment through participation in designing and administering their methods, and 
interpreting data collected through them. Further, the more extensive an array of assessment 
methods used by an institution, particularly the number of student-centered methods such as 
portfolios and capstone courses, the more likely that collected assessment information will have a 



positive impact on the institution. Comprehensive institutions should encourage or support 
alternative data collection methods within their departments if they want the resulting assessment 
data to be useful and to have a positive impact. 

Assessment Studies . Finally, once an institution has data on how its students are 
performing, do they make efforts to examine how institutional structures, policies and practices 
affect this performance? Although student learning and development are affected by students' 
internal motivation and abilities and by circumstances external to the institution, a variety of 
institutional factors can also influence how students perform, such as the availability of academic 
support resources, time spent with faculty outside of the classroom., kinds of teaching methods 
used, and interventions from advisors or counselors. About two-thirds of comprehensive 
institutions in our study do conduct such studies. Most often these studies examine the relationship 
of non-academic policies and practices (e.g., admissions standards, residence arrangement, 
financial aid and extra-curricular activities) to students’ academic performance. Comprehensive 
institutions are less likely to examine the connection between the teaching/leaming environment 
(for example, exposure to specific teaching methods, interaction with faculty) and student 
performance. The number of studies conducted by institutions is a strong predictor of using 
assessment information to make educational decisions and of achieving positive impacts on faculty 
and students alike. Yet currently, one-third of comprehensive institutions report they do not 
conduct any such studies. Clearly, these institutions may wish to commit resources to such studies 
if they want their assessment activities to produce positive results. Those institutions that do 
already conduct some studies of the influences on student performance are advised to evaluate 
whether these analyses include aspects of the teaching/leaming environment. 

Institution-Wide Assessment Support Strategy 

Comprehensive institutions have used a variety of organizational and administrative support 
strategies to support their student assessment efforts such as including student assessment within 
the academic mission; determining the main purposes for which the institution is engaging in 
assessment; sponsoring institution-wide activities to promote involvement in and support for 
assessment; adopting an institutional plan or policy for student assessment; establishing processes 
and structures for planning and coordinating assessment; and evaluating the assessment process. 
This section summarizes the institution-wide support strategies adopted by comprehensive 
institutions and offers recommendations regarding specific aspects of support strategy that enhance 
the likelihood of achieving institutional benefits from student assessment information. 

Mission Emphasis . One way to determine the degree of institutional support for an activity 
is to examine whether the activity is mentioned in its mission statement. While most comprehensive 
institutions emphasize excellence in undergraduate education, only half mention intended 
educational outcomes for students and less than one-quarter include a focus on student assessment. 
Our research shows that comprehensive institutions that have included an emphasis on 
undergraduate education and student assessment in their academic mission statement are more 
likely to use assessment information in educational decisions and to report positive impacts from 
assessment on their relationships with external constituents. Thus, leadership in these institutions 
should consider reviewing the content and emphasis of their academic mission and including 
explicit mention of student assessment if that does not currently exist. 

Assessment Purposes . Similarly, support for assessment can be examined by considering 
an institution’s purpose for engaging in student assessment. Comprehensive institutions report that 
accreditation mandates and improving undergraduate education are both very important purposes 
for engaging in assessment. Comprehensive institutions should continue to maintain such a balance 
especially as our research found that the purpose for which assessment is being conducted is linked 
to assessment information uses and impacts. Comprehensive institutions that report promoting 
internal improvement is an important purpose underlying their assessment efforts are more likely to 






actually use assessment information for institutional decisions and to improve their external 
relationships. Conversely, engaging in student assessment to meet state and regional accreditation 
requirements is not a significant predictor of assessment uses and impacts. Therefore, if 
comprehensive institutions want to benefit from their student assessment activities, they should 
examine their purposes for engaging in student assessment and be clear that these go beyond 
merely meeting external demands. 

Institution-Wide Activities . Another measure of support for student assessment is the 
number of institution-wide administrative and governance activities used to promote student 
assessment. More than half of comprehensive institution respondents report that they offer 
workshops on student assessment for academic and student affairs administrators and have a 
faculty governance committee that addresses assessment issues. However, they seldom provide 
incentives or rewards to administrators or academic units that engage in or use the results of 
student assessment, and rarely involve trustees or students in assessment committees. 
Comprehensive institutions should review the nature and number of institution-wide activities used 
to promote internal involvement in and support for assessment, and where indicated, include 
additional avenues. Not only will offering such activities demonstrate leadership support for 
assessment, but our research found that offering a greater number of administrative and 
governance activities promoting assessment is a positive predictor of the extensiveness of 
comprehensive institutions’ data collection efforts. 

Administrator and Faculty Support . Our survey also asked how supportive various internal 
constituents were of student assessment. While academic affairs and student affairs administrators 
and chief executive officers are perceived as being very supportive of assessment, trustees, faculty 
and students are viewed as being comparatively less supportive. The perceived degree of 
administrator and faculty support is positively related to the extent of student assessment 
approaches adopted by comprehensive institutions. It is advisable for comprehensive institutions to 
periodically measure the degree of internal support for assessment among internal constituent 
groups as a useful means to gauge the effectiveness of efforts to promote assessment and to 
encourage more extensive assessment activity. 

Institutional Plans and Policies for Assessment . The types of plans and policies institutions 
develop for assessing students are also indicative of their support for student assessment. Virtually 
all responding comprehensive institutions have some type of institutional plan or policy for student 
assessment in place. Most favor a decentralized approach in which academic units or programs are 
expected to assess students but determine for themselves the specific form assessment activities 
will take. However, comprehensive institutions with a formal central policy that stipulates certain 
assessment activities of all its academic programs or units are more likely to report positive internal 
impacts from assessment. This policy approach may be particularly important in comprehensive 
institutions because of their dual focus on undergraduate and graduate education, and on teaching 
and research as faculty role responsibilities. A centralized policy approach may demonstrate that the 
institution considers student assessment a priority, thus promoting internal participation in 
assessment efforts and enhancing impacts from assessment. Accordingly, comprehensive 
institutions should review the nature of their institutional plan or policy for student assessment and 
consider adopting a centralized policy if one does not already exist. 

Planning and Coordinating Assessment . The majority of comprehensive institutions have 
some type of institution-wide planning group for student assessment. Usually, these groups are 
staffed by academic administrators, faculty and institutional researchers. Responsibility for 
planning and coordinating student assessment is most often positioned within academic affairs, and 
faculty are typically represented in various aspects of the planning process. These approaches to 
planning and coordination should encourage the acceptance and use of assessment information 
within academic planning and teaching/leaming decisions. However, comprehensive institutions 
less often report involving students in assessment planning processes. Conceivably, involving 
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students in planning assessment activities would increase their understanding of the purposes of 
assessment and encourage their participation in assessment activities. Comprehensive institutions 
are encouraged to review and reconsider their internal patterns of structuring and involving various 
groups in planning and coordinating assessment efforts. 

Evaluation of Assessment Process . Whether an institution has evaluated its assessment 
approach is also an indication of the importance it accords to student assessment as an institutional 
activity. Half of the comprehensive institutions responding to our survey have not evaluated their 
assessment approach. All comprehensive institutions should consider evaluating their assessment 
plans, policies and processes in order to determine whether their efforts are meeting their intended 
objectives and whether the effort being expended to assess students is benefiting the institution. 
Further, our research found that comprehensive institutions that do evaluate their assessment 
process, either formally or informally, are more likely to use assessment information in educational 
decisions and more likely to achieve faculty impacts from assessment. 

Assessment Management Practices 

Assessment management practices refer to specific practices intentionally devised by 
institutions to manage their student assessment efforts. Four specific areas of practice were 
identified as potential influences on student assessment: academic resource allocation, student 
information systems, internal access to student information, and distribution of assessment reports 
and studies. This section summarizes the assessment management practices used by 
comprehensive institutions and provides suggestions regarding practices that maximize institutional 
uses and impacts of student assessment information. 

Academic Budget . Comprehensive institutions rarely allocate resources to academic units 
based on indicators of their student assessment activities or results. If this practice was augmented, 
institutional leaders could demonstrate that they consider student assessment to be a high priority. 

In addition, our research found that comprehensive institutions that do allocate academic resources 
based on assessment activity or results are more likely to report that their student assessment 
process produces information that is used in decision making and has a positive impact on 
indicators of student performance. Thus resource allocation practices offer a powerful way for 
comprehensive institutions to promote assessment uses and impacts. 

Computer Support . Less than half of comprehensive institutions report that they have a 
student information system that can track students from application through graduation. Even 
fewer have a system that includes student performance indicators or that is integrated with other 
institutional databases. Along with improving the management of student assessment data, creating 
relational databases in which student assessment information can be linked to other institutional 
data would facilitate conducting analyses of the relationship between students’ performance and 
their experiences within the institution. Comprehensive institutions should examine the capabilities 
of their information system for student assessment. 

Access to Assessment Information . Most comprehensive institutions do provide a variety 
of internal constituents with access to assessment information on individual students, a practice that 
encourages the use of assessment data. Providing such access makes it easier for internal personnel 
to intervene with or give feedback to students regarding their performance. We found that 
providing internal access to assessment information on individual students is a positive predictor of 
using assessment information in decisions concerning academic programs and support services. 

Assessment Report Distribution . Distributing reports that summarize assessment 
information and studies is also an important assessment management practice. Comprehensive 
institutions that distribute assessment reports to a wide array of internal and external constituents 
make greater use of assessment information in educational decisions and report more positive 



faculty and external impacts from student assessment. Report distribution helps to ensure that 
internal and external personnel are kept well informed of the institution's assessment efforts, 
student performance and institutional improvement strategies. While most comprehensive 
institutions regularly distribute assessment reports to traditional internal constituents such as 
academic administrators, faculty, and student affairs professionals, very few provide these reports 
to students, the general public or employers. Comprehensive institutions are advised to examine 
their patterns of assessment report distribution. 

Assessment Management Policies 

Assessment management policies refer to institutional policies devised to both support 
student assessment and to direct the use of student assessment information. Five content 
dimensions of assessment management policies were examined: student involvement in 
assessment; professional development on assessment for faculty and academic administrators: 
training in assessment for student affairs personnel; faculty evaluation and rewards; and academic 
planning and review processes. This section summarizes the assessment management policies used 
by comprehensive institutions and provides suggestions regarding policies that maximize 
institutional uses and impacts of student assessment information. 

Student Involvement . While comprehensive institutions report that many of their 
departments require students to participate in assessment activities and provide students with 
information regarding the purposes of these activities, few provide students with feedback on their 
assessment results or incentives for participation. Policies encouraging student involvement in 
assessment activities are important. Obviously, collecting useful student assessment information 
depends in large part upon the willingness of students to be involved in assessment activities. 
Providing explanations of the purpose of assessment activities and individual feedback to students 
regarding student performance should enhance students’ motivation to participate. Further, our 
research shows that comprehensive institutions that have developed more extensive policies on 
student involvement report making greater use of assessment information in educational decisions. 
If student involvement policies do not exist or do so only within a few departments, 
comprehensive institutions should consider extending the breadth of these policies. 

Professional Development . Comprehensive institutions do offer professional development 
on student assessment to their faculty and academic administrators. However, they offer less 
extensive policies of this nature than do other types of institutions. These policies are .more likely to 
involve activities that do not significantly affect time in the classroom (for example, funds to attend 
assessment conferences or workshops on assessment) than those that involve greater expenditure 
of administrative and faculty time (for example, paid leaves or course reduction to develop or 
improve assessment practices). While understandable from a fiscal perspective, this emphasis on 
providing professional development support of shorter duration may not provide faculty with 
adequate time to learn about or develop new assessment techniques. Offering a variety of 
professional development opportunities should increase the level of faculty and administrative 
involvement in and support for student assessment. Our research found that those comprehensive 
institutions that offer more extensive professional development on assessment to their faculty and 
academic administrators are more likely to use assessment information in educational and faculty- 
related decisions, and to report that assessment information has had a positive impact on faculty. 
Comprehensive institutions should review and consider increasing the array of professional 
development opportunities on student assessment which are offered to their academic personnel. 

Student Affairs Training . When planning professional development opportunities for 
student assessment, comprehensive institutions should not neglect their student affairs staff and 
administrators. On the whole, comprehensive institutions provide only limited training on 
assessment to their student affairs personnel, doing so in only a few departments. Yet, offering 



assessment-related training to student affairs personnel is a positive predictor of using assessment 
information in educational decisions. This policy area deserves examination. 

9 Faculty Evaluation and Rewards . Comprehensive institutions rarely have policies that 

include assessment-related criteria in faculty evaluation and rewards. Although the wisdom of this 
linkage has been debated in the assessment literature, comprehensive institutions that do include 
assessment-related criteria in their faculty evaluation and reward processes are also more likely to 
use assessment information in making educational decisions. Institutions may understandably be 
reluctant to tie faculty evaluation and rewards to assessment results or indicators of student 

• performance, but there are many other options to consider. For example, institutions can include 
criteria concerning faculty scholarship in assessment, evidence of using assessment to improve 
teaching and learning, or participating in student assessment groups or committees. 

Academic Planning and Review . Comprehensive institutions make fairly extensive use of 
student assessment information in their academic planning and review processes, particularly those 

• processes involving the review of academic departments, programs, or curriculum. Our research 
found that comprehensive institutions that incorporate assessment data in academic planning and 
review processes are more likely to use and be positively impacted by their assessment 
information. Such policies do not leave the use of assessment information to chance. Thus, 
comprehensive institutions are encouraged to build in formal linkages between the collection and 
analysis of student assessment data with specific academic planning, review and decision making 

• processes. 



Inventory ing the Institutional Student Assessment Process 

Regardless of an institution’s history of and support for student assessment, it is important 
for an institution to take stock of what it is currently doing. The Institutional Framework presented 

• in section two (Figure 1 of this monograph) provides an institutional perspective for such an 
examination. The ISSA inventory in Appendix A provides a useful quasi-objective instrument for 
identifying the specific dimensions and activities associated with the student assessment process. 

Institutions are encouraged to identify a team of faculty and administrators most 
knowledgeable about and involved with student assessment to examine their institution's activities 

• on all the dimensions in the ISSA inventory. While most items are objective (e.g., which types of 
measures are being used and how extensively), the actual pattern on campus may not be widely 
known. Other items are more subjective (e.g., the institution's purpose for engaging in student 
assessment) and can provide the basis for good discussion. 

This inventory of the institutional student assessment process then can be compared with 

• national data for comprehensive institutions which were presented in Appendix A and summarized 
in the previous sections of this monograph. This inventorying and comparison process may 
highlight activities, practices, and policies not currently used; identify areas needing greater 
attention; or focus on inconsistencies in patterns of activity (e.g., inconsistencies between stated 
purposes for student assessment and actual uses of student assessment data in academic decisions). 

• For institutions with an extensive history of involvement with student assessment, such an 
inventory may serve as a useful basis for identifying new activities to be undertaken, for improving 
the existing activities and processes, for identifying issues or controversies that need to be 
addressed, or for redesigning processes that may not be worth the current expenditure of effort and 
resources. For institutions with less experience with student assessment, the inventory may 
identify existing activities on which to build or help focus attention in the institution on the 

• importance of student assessment in improving institutional performance. In either case, the 
inventory and self-evaluation process should help both to focus faculty and administrative attention 
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on the importance of viewing student assessment as an institutional process and to deal with it 
more systematically — linking the various domains of assessment activity with institutional 
improvement. 






Planning for Student Assessment 

Student assessment in most higher education institutions has emerged, often sporadically, 
over the past decade due to the need to respond to an accreditation self study, a new state mandate, 
an academic administrator who promoted it. a faculty group who embraced it for their unit, or an 
institutional researcher or program review officer who was engaged in studies of student 
performance. While according to the data in this report, some institutions are beginning to develop 
a plan or policy for student assessment or have created a group for this purpose, there is little 
evidence of systematic planning that links student assessment to the external context and to internal 
institutional governance and management patterns; that develops organizational and administrative 
activities, practices, and policies to support it; and then uses collected assessment data for academic 
decisions and monitors their impact. Clearly there are significant institutional differences and 
complex issues to be addressed if student assessment is to have positive effects. These deserve 
some systematic, planned attention. The results of an institutional inventor}' can provide the basis 
for such an effort. 

While we do not advocate a cookbook or standardized approach to planning for student 
assessment (each institution needs to design its own planning approach to reflect its own 
governance and leadership styles and traditions), the Institutional Framework (Figure 1 in section 
two) and the institutional inventor}' provide a useful basis for redesigning or planning a student 
assessment process. Using that self-assessment, the following are planning issues that need to be 
addressed: 

External Influences 

1 . What is the nature of our state assessment process? How is it formulated and what are its 
requirements/implications for us? 

2. What are the accreditation requirements for our institutional and key professional accreditation 
bodies? 

3 . What do some of our primary external constituents expect of our graduates? 

4. What are external sources of support (educational, financial, technical) for our student 
assessment efforts? 

Institution-Wide Support Patterns 

1 . What emphasis is placed on student assessment in our mission statement? What are the 
intended purposes of our institution’s assessment efforts? 

2. What institution-wide administrative, governance, and academic activities support student 
assessment? Who are key leadership support groups and how are they involved? 

3 . What is the nature of our institution-wide plan or policy for student assessment? The role and 
membership of a coordinating or planning body? And the pattern of authority and 
responsibility for administering the process? 

4. Is the student assessment process to be evaluated? By whom? Using what criteria? 
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Approaches to Student Assessment 



1 . Should we have an institution-wide or decentralized (by academic unit) approach to student 
assessment? 

2 . What types of measures are to be used? How widely? At what points in time ? 

3 . What types of instruments and methods are appropriate? What technical or professional 
support does that require? 

4. What studies of the influence of students’ educationally-related experiences on their 
performance are to be conducted? What reports of student performance are to be prepared and 
distributed? 

Assessment Management Practices and Policies 

1 . What assessment management practices exist to guide student assessment? (e.g.. resource 
allocation practices, information systems, data access practices, and report distribution 
practices) 

2. What institutional policies promote the use of student assessment? (e.g., policies on student 
involvement, professional development, student affairs training, faculty evaluation, and 
academic planning) 

Uses and Impacts of Student Assessment 

1 . How will we assure the use of student assessment information in educationally- and faculty- 
related academic decisions? 

2. How will we monitor the impact of student assessment on our students, faculty, academic and 
instructional patterns, and on our external relationships? 

. While planning for student assessment may not resolve all of these questions, they should 
be addressed as should issues of the balance between the effort and resources required to maintain 
the institution’s student assessment process and the educational and institutional benefits derived 
from assessment. When planning is combined with inventorying, these two processes can become 
a powerful mechanism for understanding and improving institutional student assessment 
endeavors. Inventorying existing student assessment processes is a first step toward understanding 
the nature and extent of an institution’s student assessment activities. This can be followed by a 
planning process that considers the recommendations for comprehensive institutions that were 
presented in this section. Using a systematic planning approach should increase the likelihood that 
student assessment will contribute to improved institutional performance. 



O 

ERIC 



69 



77 



APPENDICES 
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An Introduction to the ISSA 



The Institutional Support for Student Assessment Inventory (ISSA) was developed as pan of a national research 
program examining the Organizational and Administrative Support for Student Assessment for the National Center 
for Postsecondary Improvement (NCPI). The ISSA is designed as an institutional inventory of the organizational and 
administrative practices that have been designed and implemented to support the use of student assessment on vour 
campus. . • 

Institutional Support Practices are those organized activities, policies, and procedures that your institution has 
intentionally designed to enhance the practice of student assessment. Student Assessment refers to those activities 
focused on measuring dimensions of student performance other than traditional end of course grading. 

This national survey is designed to identify institutional support practices for undergraduate student assessment. The 
project also examines the factors influencing the adoption of various support practices and how those practices 
enhance the impact of student assessment for institutional improvement. 

We understand that being selected for this survey will require a commitment of time to complete and we appreciate 
your involvement. This instrument is also intended as an institutional self-assessment inventory to facilitate 
examination of your institution’s own organizational and administrative practices which support student assessment. 
We encourage each institution to use the survey in this manner. You will receive a summary report of survey 
responses to all compare with your own institutional profile. 

Completing the ISSA 

The main purpose is to obtain a profile of your institution's current approach to undergraduate student assessment 
and its support practices. The inventory may be completed by one individual or group of individuals who are most 
familiar with the patterns of undergraduate student assessment on your campus. It should take less than one hour to 
complete. 

• Please keep in mind that the questions refer to undergraduate education at your institution. 

• Respond to each item in the questionnaire to the best of your knowledge. 

The questionnaire is coded to allow follow up only. Individual institutions will not be identified in any analyses or 
reports. 

Return the completed questionnaire in the enclosed return envelope. Any questions concerning the survey can be 
addressed to the, following: 

National Center for Postsecondary Improvement Project 5.2 
School of Education 
University of Michigan 
610 E. University, Room 2339 
Ann Arbor, MI 48109-1259 
Phone: 734-647-2464 
Fax: 734-936-2741 
Email: ncpi.proj52@umich.edu 



Marvin W. Peterson, Project Director 

NCPI is funded by the U.S. Department of Education’s Office of Educational Research and Improvement under OERI 
grant number R309A60001 

©1997. The Regents of the University of Michigan 



I. Institutional Approach to Student Assessment 



A . Type, Extent and Timing of Student Assessment 



We are interested in your institution's routine practices of collecting different types of undergraduate student 
performance data, the extent to which they are collected, and when they are collected. For each of the following 
content types of undergraduate student performance data: 

1) indicate the extent to which each type is collected 

2) for each type of data collected, check whether it is collected at entry, during enrollment, at exit, or a 
combination of these data collection points. 



Tvpe 


Extent 




Timing 




Currently Enrolled Students 


Collected 


Collected Collected 


Collected 


Collected 


Collected 




Not for some 


for many for all 


at 


while 


at 




Collected students 


students students 


entry 


enrolled 


exit 




(circle one number for each item) 


(check all that apply for 


each item) 




1 2 


3 4 










CMP 


All 








1. Student academic intentions or 












expectations 

2. Basic college-readiness 


2.99 


3.25 








skills (reading, writing, 
mathematics, etc.) 


3.40 


3.44 








3. Higher-order skills (critical 
thinking, problem solving) 


2.25 


2.10 


■ 


— 


— 


4. General education competencies 


2.61 


2.55 








5. Competence in major field 
of study (discipline- or 
program-specific knowledge) 


2.86 


2.60 


— 


— 


— 


6. Vocational or professional skills 


2.00 


2.1 1 








7. Personal growth and 
affective development 
(values, attitudes, social 
development, etc.) 


2.29 


2.12 








8. Student experiences and 












involvement with institution 
9. Student satisfaction with 


2.68 


' 2.57 








institution 


2.97 


2.96 









10. Student academic progress 

(retention, graduation rates) 3.86 3.76 

Former Students 



1 1 . Vocational or professional 
outcomes (career goals, job 
attainment or performance) 

12. Further education 
(transfer, degree attainment, 
graduate study) 

13. Civic or social roles (political, 
social or community involvement) 

14. Satisfaction and experiences 
with institution after leaving 



2.75 2.72 

2.71 2.69 

2.08 1.80 

2.75 



2.63 



B. 



Student Assessment Instruments 



Does your institution employ institutionally or externally developed instruments or tests for the following types of 
undergraduate student assessment information? (circle all that apply for each item): 








Source of 


Instrument 






Not 


Institutionally 


State 


Commercial! 


Content of Instrument 


used 

1 


developed 

-> 


provided 


available 

4 


1. Student plans, goals, or expectations 


CMP: 23% 


46% 


3% 


39% 




All: 2 1 % 


51% 


4% 


32% 


2. Basic college-readiness skills 


CMP: 5% 


53% * 


12% 


59% 


(reading, writing, mathematics, etc.) 


All: 6% 


38% 


1 1% 


67% 


3. Higher-order skills 


CMP: 34% 


33% 


2% 


42 % * 


(critical thinking, problem solving) 


All: 42% 


29% 


2% 


32% 


4. General education competencies 


CMP: 31% 


42% 


■5% 


35% 




All: 32% 


40% 


5% 


33% 


5. Competence in major field of study 


CMP: 9%* 


76% * 


16% 


57 % * 


(discipline- or program-specific knowledge) 


All: 19% 


64% . 


12% 


39% 


6. Vocational or professional skills (excluding 


CMP: 37% 


43% 


19% 


29% 


licensure exams) 


All: 34% 


43% 


14% 


25% 


7. Personal growth and affective development 


CMP: 29% 


46% 


3% 


39% 


(values, attitudes, social development, etc.) 


All: 38% 


39% 


2% 


30% 


8. Student effort, experiences or involvement 


CMP: 13% 


66% 


4% 


31% 


with institution 


All: 21% 


60% 


4% 


24% 


9. Student satisfaction with institution 


CMP: 3% 


71% 


9% 


44% 




All: 4% 


73% 


8% 


35% 


10. Alumni satisfaction and experiences 


CMP: 4% 


86% 


9% 


18% 




All: 10% 


78% 


8% 


15% 



c. 



Other Student Assessment Methods 



• 


To what extent does your institution use the following methods to collect undergraduate student assessment 
information? (circle one number for each item ): 




Other Student Assessment Methods 


Not 

used 

1 


Used in 
some units’ 
-> 


Used in Used in 

most units all unit> 

3 4 


• 


1 . Observations of student performance 
(simulations, demonstrations, lab) 




CMP 

2.24 


All 

2.26 




' 2. Student portfolios or comprehensive projects 




2.18 


2.10 


• 


3. Student performance in capstone courses 




2.41 


2.15 




4. Student interviews or focus groups 




1.98 


1.84 




5. Transcript analysis 




2.13 


2.16 


• 


6. External examination of students 
(licensure exams, external reviewers) 




1.08* 


2.02 




7. Special surveys of or interviews with 
withdrawing students 




2.35 


2.40 


• 


8. Alumni interviews or focus groups 




2.03 


1 .90 




9. Employer interviews or focus groups 




1.87 


1.87 


• 


D. Student Sub-Populations 










Does your institution use different assessment methods for the followin 
(check one for each item): 

Different 


g sub-populations of undergraduate students*? 

Same as Other Students 


• 


1 . Adult students 

2. Part-time students 

3. Minority students 

4. Distance education students 


CMP 

16% 

5% 

'3% 

25% 


All 

10% 

5% 

2% 

22% 








* “Unit” refers to academic areas such as departments, divisions, schools, or colleges. 



►ERIC 



75 



.83 



E. Student Assessment Studies 



Does your institution conduct studies of the relationship between the following experiences and students* 
performance (check ah that apply): 





CMP 


All 


1. Student course-taking patterns 


24% 


269; 


2. Exposure to different instructional or teaching methods 


19% 


219 ; 


3. Patterns of student-faculty interaction 


13% 


14% 


4. Extra-curricular activities 


30% 


O' 

’-r 
r i 


5. Residence arrangements 


32% * 


:i < 7 ( 


6. Student financial aid and/or concurrent employment 


31% 


30 c i< 


7. Admission standards or policies 


52% * 


429r 


8. Academic advising patterns 


29% 


269! 


9. Classroom, library and/or computing resources 


14% 


179; 


10. Do not study the relationship between the above experiences 


and student performance 35% 


389; 



Does your institution provide profiles or reports of appropriate student performance 


information at the foil 


owing 


levels of aggregation (check ad that apply): 


CMP 


All 


1 . Institution wide 


74% 


699; 


2. Schools or colleges 


54%* 


31% 


3. Academic programs or departments 


76% * 


65% 


4. Special populations or subgroups/studenls 


51% 


46% 


5. By course or groups of courses 


27% 


36% 


6. Do not provide any reports 


7% 


1 1% 



II. Institutional Support for Student Assessment 
A. Institutional Emphasis 

1 . Your institutional mission statement explicitly (check ad that apply): 





CMP 


Ad 


a. emphasizes excellence in undergraduate education as an institutional priority 


89% 


82% 


b. identifies the educational outcomes intended for your students 


53% 


52% 


c. refers to student assessment as an important institutional activity 


22% 


I97r 


d. does not explicitly mention any of the above 


7% 


1 1%: 



For how many years has your institution engaged in student assessment? 



2 . 



B. Purpose of Student Assessment 

The following are often intended purposes of an institution's undergraduate student assessment process. Please rate 
the importance of each for your institution, (circle one number for. each item): 



Purpose 


No 


Minor 


Moderate 


Ver\ 




Importance 

1 


Importance 

2 


Importance 


Important 

4 






CMP 


All 




1. Preparing institutional self-study 
for accreditation 

2. Meeting state reporting 




3.67 


3.86 




requirements 

3. Guiding internal resource 




2.76 


2.89 




allocation decisions 
4. Guiding undergraduate academic 




2.62 


2.71 




program improvement 
5. Improving the achievement of 




3.46 


3.43 




undergraduate students 




3.50 


3.48 




6. Improving faculty instructional 
performance 

7. Other f brief! v describe): 




2.95 


3.02 





C. Administrative and Governance Activities 



Institutions have introduced a variety of administrative or governance activities that address or promote student 
assessment. Does your institution engage in any of the following activities? (check all that apply): 



CMP 

1 . Annual presidential or other institution-wide initiatives, forums or seminars on 



assessment 41% 

2. Rewards or incentives for academic and student affairs administrators who promote 

use of student assessment in their unit 7% 

3. Incentives for academic units to use student assessment information in their 

evaluation and improvement efforts 31 % 

4. Student assessment workshops for academic and student affairs administrators 61 % 

5. Board of trustees committee that addresses student assessment 1 7 % 

6. Faculty governance committee that addresses student assessment issues 64 % 

7. Student representation on student assessment committees 41 % 



All 



41% 



6 % 



27% 
56% 
1 3% 
58% 
33% 



ERIC 



77 



85 



D. Support for Student Assessment 



Use the scale below to rate the degree to which various groups within your institution support undergraduate student 
assessment activities (circle one number for each item): 





Very 

Unsupportive 

i 


Somewhat 
Unsupportive 
-> • 


Neutral. 

Unknown 

;% 

ALL 


Q 


i 


CMP 


1 . Board of trustees 




3.90 


3.84 


2. Chief executive officer 




4.42 


4.41 


3. Academic affairs administrators 




4.68 


4.64 


4. Student affairs administrators 




4.27 


4.33 


5. Faculty governance 




3.77 


3.80 


6. Students 




3.24 


3.33 



Supportive 

4 



Ver> 

Supportive 



E. 



Planning and Coordinating Student Assessment 



1 . Which of the following best describes your institution's plan or policy for undergraduate student 

assessment? Your institution (check afl that apply): 



b. 



c. 



e. 



has a formally adopted institutional plan or policy requiring specified 
undergraduate student assessment activities of ad academic units or programs 
has a formally adopted plan or policy for undergraduate student assessment 
in some academic units or program areas (e.g. general education or 
academic majors) 

has a formally adopted institutional plan or policy requiring all academic units or 

programs to develop their own undergraduate student assessment plan 

has a formally adopted institutional plan or policy stipulating institution-wide 

activities to be conducted by a central committee, office, or officer 

has no formal plan or policy but academic units or programs are encouraged to 

conduct their own undergraduate student assessment activities 

is currently developing a plan or policy for undergraduate student assessment 

does not have an undergraduate student assessment plan or policy (skip TO E-6) 



CMP 


All 


47% 


50% 


21% 


19% 


59 % * 


39% 


40 % 


38% 


15% 


1 3% 


21% 


17% 


2% 


4% 



2 . 



Is there an institution-wide group (committee, task force, etc.) that is primarily responsible for ongoing 
planning and policy setting for undergraduate student assessment? (check one ): 



a. yes 

b. no (skip to Question E-5) 



CMP All 

73% 70% 



ERIC 



78 



86 



3. If yes. who serves on this group? (check aH that apply t: 







CMP 


All 


a. 


Chief executive officer 


7% 


1 3% 


b. 


Academic affairs administrators )/staff 


87 % 


86% 


c. 


Student affairs adminislralor(s)/staff 


47 7c 


54% 


d. 


Institutional research adminislralor(s)/slaff 


59 % 


61% 


c. 


Academic review and evaluation administrators )/staff 


227c 


24% 


f. 


Student assessment adminislralor(s)/staff 


32 7c 


32% 


o 
e * 


Faculty 


93% 


91% 


h. 


Students 


38% 


33% 


i . 


Other 


8% 


1 2% 



Who has executive responsibility for or who chairs the institution- 


■wide group responsible lor the 


ongoin 


pi, 


anning or policy-setting process for undergraduate student assessment? (check that apply): 








CMP 


All 


a. 


Academic affairs administrator 


54 % 


55% 


b. 


Student affairs administrator 


3% 


17c 


c. 


Institutional research officer 


1 7 % 


18%' 


d. 


Academic review and evaluation officer 


6% 


5% 


e. 


Student assessment officer (if separate) 


8% 


oc 

* 


f. 


Faculty member 


38 % 


3 1 % 


CT 
C ' 


Other 


11% 


1 1% 



5. Who approves any changes in your institution’s plan or policies for undergraduate student assessment? 
(check a]] that apply): 







CMP 


All 


a. 


Board of trustees 


12 % 


1 7%' 


b. 


Chief executive officer 


37 % 


45% 


c. 


Chief academic affairs officer 


80% 


157c 


d. 


Chief student affairs officer 


1 3 % 


20 7c 


e. 


Institutional research officer 


1 57c 


18% 


f. 


Academic review and evaluation officer 


8 % 


8% 


a 

c* 


Student assessment officer 


9% 


10%- 


j- 


Student government 


4% 


1 % 


h. 


Academic senate or other faculty commitlec(s) 


48 % 


39% 


i. 


Faculty union 


5% c 


47c 


k. 


Other 


11% 


14 7c 


Who has operational responsibility for your institution's 


day-to-day undergraduate student assessment 


activities (e.g., instrument development, data collection, analysis, and reporting)? (check aM that apply): 






CMP 


All 


a. 


Academic affairs administrator 


44 % 


45% 


b. 


Student affairs administrator 


16% 


20% 


c. 


Institutional research officer 


47 % 


45% 


d. 


Academic review and evaluation officer 


11% 


9% 


e. 


Student assessment officer 


15 % 


157c 


f. 


Faculty member(s) 


39 % 


33% 


g. 


Other 


16% 


13% 


h. 


No one (SKIP TO QUESTION E8) 


3% 


3% 



To whom does ihe individual wiih operational responsibility for day-to-dav student assessment activities 
directly report? (check one): 







CMP 


All 


a. 


Chief executive officer 


199c* 


29 7< 


b. 


Chief academic officer 


11 7c* 


569r 


c. 


Chief student affairs officer 


3 7c 


17c 


d. 


Institutional research officer 


4 7c 


3% 


e. 


Academic review and evaluation officer 


\7c 


29r 


f. 


Other 


11 7c 


1 09i 



Is there an office which provides faculty consultation in using student assessment for instructional 
improvement or cumculum development? (check one ): 







CMP 


All 


a. 


yes 


53 7c 


479r 


b. 


no 


417c 


539r 



If yes, what is the name of the office?. 



Evaluating Your Institution's Student Assessment Plan or Process 

Has your institution evaluated its undergraduate student assessment process? (check one): 







CMP 


All 


a. 


yes, with a formal evaluation 


19% 


22% 


b. 


yes, with an informal evaluation 


30% 


27% 


c. 


currently developing evaluation plans (SKIP TO SECTION III) 


31% 


29% 


d. 


not currently evaluating or planning to evaluate assessment process 
(SKIP TO SECTION) 


19% 


21% 



In evaluating your institution’s student assessment process, which of the following elements of that 
process were reviewed? (check all that apply): 







CMP 


All 


a. 


your student assessment plan and policies 


79% 


81% 


b. 


the structure and responsibility for student assessment 


68% 


64% 


c. 


achievement of your institution's intended objectives for student 








assessment 


74% 


70% 


d. 


reliability and validity of student assessment instruments and methods 


52% 


54% 


e. 


quality of data analysis 


54% 


51% 


f. 


use of student assessment information in institutional decision-making 


60% 


66% 


g- 


the problems encountered while conducting student assessment activities 


74% 


69% 


h. 


comparison of the costs and benefits of student assessment 


23% 


22% 



III. External Influences on Institutional Student Assessment Activities 



A . State Role (FOR STATE-FUNDED INSTITUTIONS ONLY: ALL OTHERS SKIP TO QUESTION III. B-l ) 
I . Was your state’s plan/requirement for student assessment primarily developed (check one)): 







CMP 


All 


a. 


by stale-level officials 


169c 


1 69 


b. 


through joint consultation between stale officials and institutional 








representatives 


32 9c 


399 


c. 


no statewide plan or requirement for student assessment exists (SKIP TO III. B-l 


) 519c 


469 



Slate requirements for student assessment (check Ml that apply): 










CMP 


All 


a. 


were an important reason for your institution to initiate undergraduate student 








assessment 


419c 


459 


b. 


have increased your institution’s involvement in undergraduate student 








assessment 


569 


629 


c. 


have not been a factor in your institution's undergraduate student assessment 








activities 


30 9c 


229c 


d. 


have been a negative influence on your institution's undergraduate student 








assessment activities 


69c 


49c 


Your slate’s reporting requirements include (check Mi that apply): 










CMP 


All 


a. 


evidence that a student assessment plan is in place 


63 9c 


689 


b. 


measurement of state-mandated student performance indicators 


66 9c 


649c 


c. 


institutionally-devised student performance indicators 


50 9c 


499c 


d. 


evidence of institutional use of student assessment information 


48 9c 


529c 



4. How has your stale higher education agency reviewed or evaluated your institution's undergraduate student 

assessment plan or process after it was implemented? (check Ml that apply): 







CMP 


All 


a. 


reviewed by state officials 


36 9c 


429r 


b. 


reviewed using external reviewers 


129c 


169c 


c. 


required an institutional self-review 


179c 


249c 


d. 


no post hoc review has occurred (SKIP TO QUESTION B- 1 ) 


54 9c* 


449c 


The state review of your institution’s undergraduate student assessment plan 


or process included (check Mi 


that apply): 










CMP 


All 


a. 


review of your institution’s student assessment process itself 


70 9c 


67% 


b. 


comparison of your institution’s student performance record with 
your past performance 


529c 


44% 


c. 


comparison of your institution’s student performance record with 
peer institutions 


35 9c 


36% 


d. 


comparison of your institution’s student performance record with 
institutions in your state 


389c 


38% 


e. 


other (briefly describe) 


79c 


10% 



B. Regional Accrediting Role in Student Assessment 

I . Has your institution gone through a regional self study accreditation review which required undergraduate 
student assessment? (check one ): 







CMP 


All 


a. 


yes 


82% 


809; 


b. 


no 


18% * 


29% 



Regional accreditation agency requirements for undergraduate student assessment (check aH that apply): 



CMP All 

a. were an important reason for your institution to initiate undergraduate student 

assessment 72% 64% 

b. have increased your institution's involvement in undergraduate student 

assessment 85 % 7991 

c. have not been a factor in your institution's undergraduate student assessment 

activities 8% I29< 

d. have had a negative influence on your institution's undergraduate std. 

assessment activities 2% 0.9 7c ‘ 



3. Your institution's regional accreditation agency requires (check a U that apply): 



CMP All 

a. evidence that a student assessment plan or process is in place 94 % 909r 

b. intended institutional uses of student assessment information 76% 73% 

c. results of student assessment 69% 669 r 

d. evidence of actual institutional use of student assessment information 80% 77% 

e. unfamiliar with regional accreditation requirements for student assessment 4% 5% 



External Sources of Support for Assessment 
Has your institution received external grants to improve underg 


raduate student assessment practices from any 


the following? (check ah that apply): 


CMP 


All 


a. F1PSE 


9% 


6% 


b. other federal agencies (please identify): 


3% 


1 % 


c. a stale incentive program 


10% 


7% 


d. private foundations or corporate sources (please identify): 


6% 


6% 


e. no known external grants received 


77% 


79% 



2. Has your institution used any of the following student assessment services offered by the following 
postsecondary organizations? (check aH services that apply for each type of organization): 



Type of Postsecondary Organization 



a. Professional associations (Institutional, 
disciplinary, or administrative) 



b. Regional accrediting association 



c. Stale-level agency 



d. Consortium of institutions 





Student 


Assessment 


Service Used 












Publications 


Noi used or 


Consultation ■ Assessment 


Training 


or research 


not available 


services 


conferences 


workshops 


reports 


CMP: 20% 


20% 


60% 


36% 


61 %* 


All: 29% 


13% 


5 1 % 


32% 


5 1 % 


CMP: 19% * 


20% 


47% 


39% 


56 % * 


All: 30% 


19% 


41% 


32% 


45% 


CMP: 57% 


8% 


27% 


16% 


23% 


All: 54% 


14% 


26% 


22% 


22% 


CMP: 51% 


10% 


35% 


16% 


21 % 


All: 53% 


13% 


30% 


18% 


20% 



IV. Academic Management Policies and Practices for Student Assessment 

Institutions have a wide array of formally organized policies, activities, and procedures intended to enhance or support 
the collection and use of undergraduate student assessment information. The following policies and practices have 
been identified in many institutions. 

For Questions a through D, indicate whether the following policies or practices exist at your 

INSTITUTION. 



A. Resource Allocation for Student Assessment (check all that apply): 



1. An explicit operating budget allocation is made to support student assessment. 

2. An academic budget process that considers student performance indicators in resource 

allocation to academic units. 

3. An academic budget process that compares academic units on student performance 

indicators and allocates resources competitively. 

4. An academic budget process that rewards academic units for improvement based on 

their own past student performance indicators. 



C M P 
53% 

25% 

2 % 

5% 



All 

49% 

23% 

2 % 

3% 



O 

ERJC 



83 



91 



B. Student Assessment Information System (check ah that apply): 





CMP 


All 


1 . Key student assessment activities have been scheduled into the academic calendar. 

2. A computerized student information system which includes student performance 


56% 


57% 


indicators 

3. Student information system tracks individual students from application through 


24% 


28% 


graduation. 


43% 


42% 


4. Student assessment database integrated with faculty, curricular. & financial database 


s. 8% 


10% 



C. Access to Individual Student Assessment Information (check all that apply): . 



Student assessment information on individual students is available to: 





CMP 


Ail 


1. Institutional research, assessment or evaluation professionals 


77% 


76% 


2. Senior academic administrators 


69% 


72% 


3. Department chairs or academic program administrators 


76% 


73%- 


4. Student affairs professionals 


49% 


58% 


5. Faculty advisors 


63% 


66% 



D. Distribution of Student Assessment Reports and Studies (check ah that apply): 
Student assessment reports and studies or appropriate summaries are regularly distributed to: 





CMP 


Ail 


1 . Students 


17% 


19% 


2. Faculty 


68% 


67% 


3. Academic administrators 


87% 


86% 


4. Student affairs professionals 


52% 


58% 


5. Employers 


4% 


5% 


6. The general public 


7% 


8% 



For Questions E through h. use the following scale to indicate the extent to which each of the 

FOLLOWING POLICIES AND PRACTICES EXIST AT YOUR INSTITUTION (Circle one number for each item). 





Not 


Done in 


Done in 


Done in 


Done 




done 


a few 


some 


many 


i n 


E. Student Policies on Student Assessment 


at ail 


depts. 


depts. 


depts. 


most 




I 




3 


4 


depts. 

5 






CMP 




All 




1 . Students are required to participate in student assessment 
activities 




3.66 




3.77 




2. Incentives are provided to encourage students to 
participate in student assessment activities 




1.91 




1.87 




3. Information regarding the purpose and uses of student 
assessment is provided to students 

4. Students are provided with individual feedback regarding 




3.42 




3.52 




their own student performance results 




2.97 




3.21 





84 



F. Professional Development 

1 . Faculty are required to learn about or receive training on 
student assessment 

2. Funds for faculty to attend or present at professional 
conferences on student assessment are available 

3. Workshops, seminars, or consultative services for 
faculty on the use of student assessment in course 
design or instruction are offered 

4. Assistance for faculty in the form of paid leaves, 
stipends, mini grants or course reduction to improve use 
of student assessment is provided 

5. Workshops and seminars for department chairs, deans, 
and other academic administrators to improve use of 
student assessment in their unit is provided 

6. Student affairs staff are required to learn about or receive 
training related to student assessment 

7. Student assessment workshops for student affairs 
administrators are provided 



G. Faculty Evaluation and Rewards 

1. Faculty evaluation for promotion considers evidence of 
student performance in their classes (not just student 
teaching evaluation) 

2. Faculty evaluation for annual salary and merit increases 
incorporates evidence of student performance 

3. Faculty scholarship on or innovative uses of student 
assessment is considered in promotion, tenure, or salary 
reviews 

4. Faculty willingness to use or to participate in student 
assessment activities is considered in faculty promotion, 
tenure, or salary reviews 

5. Faculty receive public recognition or awards for 
innovative or effective use of student assessment 

6. Faculty hiring process considers experience or skill in 
student assessment 

7. Faculty are encouraged to assess student learning in their 
classes 



H. Academic Planning and Review 

Your institution incorporates student performance data into 
the following processes: 

I. Academic department or undergraduate program planning 
or review 

2. General education or core curriculum review 

3. Course-level review and development 

4. Review and planning for student academic support 
services 



Not 


Done in 


Done in 


Done in 


Done 


done 


a few 


some 


man> 


i n 


at all 


depts. 


depts. 


depts. 


111 ost 


1 




3 


4 


depts. 










s 




CMP 




All 






3.7 8 




3.67 






3.42 




3.55 






3.24 




3.36 






2.92 




3.09 





V . Impacts of Student Assessment 



A. Decision Making 



To what extent has the use of information available from your undergraduate student assessment process influenced 
the following actions? (circle one number for each item): 





No action or 


Action taken. 


Action taken. 


Action taken 


Institutional Actions 


influence 


data not 


data somewhat 


data very 




unknown 

1 


influential 

2 


influential 

3 


influential 

4 






CMP 


All 




1. Revising your undergraduate academic 










mission or goals 




2.16 


2.06 




2. Designing or reorganizing academic 










programs or majors 




2.67 


2.54 




3. Designing or reorganizing student 








' 


affairs units 




1.90 


1.91 




4. Allocating resources to academic units 




1.79 


1.81 




5. Modifying student assessment plans. 










policies, or processes 




2.60 


2.61 




6. Deciding faculty promotion and tenure 




1.45 


1.46 




7. Deciding faculty salary increases or 










rewards (release time, travel funds, etc.) 




1.45 


1.39 




8. Revising or modifying general 










education curriculum 




2.55 


2.47 




9. Creating or modifying student out- of- 










class learning experiences (e.g. 










internships, service learning) 




2.22 


2.14 . 




10. Creating or modifying distance 










learning initiatives 




1.70 


1.72 




1 1. Modifying instructional or teaching 










methods 




2.51 


2.47 




12. Modifying student academic support 










services (e.g. advising, tutoring) 




2.56 


2.56 
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B. Institutional Impacts 



Have you monitored the following institutional indicators and been able to document the impact of student 
assessment information on them? (circle one number for each item): 





Not monitored. 


Monitored. 


Monitored. 


Monitored. 


Internal Impacts 


do not 


negative 


no known 


positive 




know 

1 


impact 


impact 


impact 




J 


CMP 


All 


-t 


1 . Affected campus discussions of 
undergraduate education 




2.41 


2.28 




2. Contributed to faculty satisfaction 

3. Contributed to faculty interest in 




1.60 


1.69 




teaching 

4. Led to changes in instructional or 




1.89 


1.88 




teaching methods used 




2.46 


2.45 




5. Contributed to student satisfaction 

6. Affected student retention or 




2.04 


2.03 




graduation rates 




2.15 


2.20 




7. Affected student grade performance 

8. Affected student achievement on 




1.80 


1.95 




external examinations (e.g. 
professional licensure, GRE) 




1.94 


1.97 




External Impacts 




CMP 


All 




9. Affected student applications or 










student acceptance rates 




1.50 


1.48 




10. Affected allocation or share of state 










funding 

1 1. Affected evaluation from regional 




1.43 


1.46 




accreditation agency 




2.66 


2.55 




12. Affected private fund-raising results 




1.44 


1.42 




13. Affected success on grant 
applications 




1.56 


1.65 




14. Affected communication with 










external constituents 




1.81 


1.75 




15. Affected institutional reputation or 










image 




1.99 


1.94 





Note: * denotes difference greater than or equal to .3 or 10% from average of all institutions. 
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VI. Further Studies - Optional 



This page will be removed from the questionnaire before it is processed and completion of it is optional. However, 
we would like to know more about your institution's experience with student assessment and we would like to be 
able to respond to you personally with a follow up report. 



Within the next year several institutions will be invited to participate in a more intensive study of the impacts of 
their student assessment practices and policies. Would you be interested in participating in a case study? 

yes 

possibly 

no 

If you are interested, we would appreciate any additional information regarding your student assessment practices that 
you believe would be of interest to other institutions. If you believe your approach to student assessment or its 
impacts are unusual, please describe it briefly (or enclose a report you think captures vour experience). 



Please provide your name and address if you are interested in receiving a personal summary report of this survey. 

Name: 

Title: 

Institution: 

Address: 



Phone: 

E-Mail: 



Thank you for taking the time to complete this instrument. 
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Appendix B 



/ 



Construction and 
Content of Variables 



o 
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Variable Definition 



Institutional Characteristics 




Enrollment 


Single item. Reflects number of students enrolled in institution. Data from 1995 
Integrated Postsecondarv Education Data System (IPEDS). 


Control 


Single item. (1 = public: 0 = private). Data from IPEDS. 


Institutional type 


Four dummy-coded single items. Reflects the institution's Carnegie type. 
(Associate of Arts. Baccalaureate. Doctoral, and Research. Master' s institutions 
was the omitted category.) Data from IPEDS. 



External Influences on Student Assessment 

Slate initiative Single item. Reflects whether the state's assessment initiatives were euided bv 





legislative or other means (1 = no state plan: 2 = state policy; 3 = state statute; 4 
= combination of policy &. statute). Data from SAS. . 


State approach 


Single item. Reflects whether stales mandate common indicators and outcomes (1 
= no indicators or outcomes; 2 = institutional specific: 3 = common for some: 4 
= common for all). Data from SAS. 


Accrediting association 


Five dummy-coded single items. Reflects the institution's regional accreditation 
association membership (Middle States; North Central; New England; Southern: 
Western. Northwest region was the omitted region). Data from IPEDS. 


Development of state plan 


Single item. (Ill A 1) Reflects how state plan for student assessment was 
primarily developed (1 = state; 2 = joint consultation between slate and 
institution; 3 = no state plan or requirement). 


State influence 


Four single items. (Ill A 2 a-d) Reflect the influence of state requirements on the 
institutions assessment activities; a = important reason to initiate student 
assessment; b = increased institution's involvement in assessment; c = have not 
been a factor in assessment activities: d = have been negative influence on 
assessment activities ( 1 = yes; 0 = no). 


State reporting requirements 


Four single items. (Ill A 3 a-d) Reflect the slate's reporting requirements: a = 
evidence that assessment plan is in place: b = measurement of state mandated 
indicators; c = use of institutionally devised indicators: d = evidence of 
institutional use of assessment information ( 1 = yes; 0 = no). 


Slate review methods 


Four single items. (Ill A 4 a-d) Reflect the method used by stale to review the 
institutions assessment activities: a = reviewed by stale officials; b = reviewed 
using external reviewers; c = required institutional self-review; d = no review 
occurred ( 1 = yes; 0 = no). 


State review criteria 


Five single items. (Ill A 5 a-e) Reflect the processes included in the state review 
of the institutions assessment activities: a = review of institutions process itself; 
b = compare student performance record with past record; c = compare student 
performance record with peer institutions; d = compare student performance record 
with other in state; e = other ( 1 = yes; 0 = no). 



Variable 


Definition 


Accrediting influence 


Four single items. (Ill B 2 a-d) Reflect the influence of regional accreditation 
agency requirements on the institutions assessment activities: a = important . 
reason to initiate student assessment: h = increased institution's involvement in 
assessment: c = have not been a factor in assessment activities: cl = have been 
negative influence on assessment activities (1 = yes: 0= no). 


Accrediting reporting 
requirements 


Five single items.* (Ill B 3 a-e) Reflect the regional accreditation agency reporting 
requirements: a = evidence that assessment plan is in place: b = intended uses of 
assessment information: c = results of assessment: d = evidence of actual 
institutional use of assessment information: e = unfamiliar with regional 
accreditation requirements ( 1 = yes: 0 = no). 


External sources of support 


Five single items. (Ill C 1 a-e) Reflect the sources of support received to improve 
student assessment practices: a = FIPSE: b = other federal agencies: c = state 
incentive program: d = private foundation or corporate source; e = no known 
external grants ( 1 = yes; 0 = no). 


Use of external services 


Four single items. (Ill C 2 a-d) Reflect the use of services offered by each of the 
following type of postsecondarv organization: a = professional associations; b = 
regional accrediting association; c = state-level agency: cl = consortium of 
institutions. Respondents could choose from the following services offered by 
each organization: organization not used or not available: consultation services: 
assessment conferences: training workshops; publications or research reports ( 1 = 
used; 0 = not used). 



Institutional Approach to Student Assessment 

Academic inieniions Single item. (I A 1) Reflects extent to which institutions collect data on current 





student’s academic intentions or expectations (I = not collected; 2 = collected for 
some students; 3 = collected for many students; 4 = collected for all students). 


Basic college-readiness skills 


Single item. (I A 2) Reflects extent to which institutions collect data on current 
student’s college-readiness skills (1 = not collected: 2 = collected for some 
students; 3 = collected for many students: 4 = collected for all students). 


Cognitive assessment 


Four item factorially-derived scale. (I A 3-6) Reflects the extent to which 
institutions collect data on current students' cognitive performance: competence 
in major Held; general education competencies; higher-order cognitive skills; 
vocational or professional skills (1 = not collected; 2 = collected for some 
students; 3 = collected for many students; 4 = collected for all students). Cronbach 
alpha = .71. 


Affective assessment 


Three item factorially-derived scale. (I A 7-9) Reflects the extent to which 
institutions collect data on current students' affective development and 
satisfaction: experiences and involvement with institution; satisfaction with 
institution; personal growth and affective development ( 1 = not collected; 2 = 
collected for some students; 3 = collected for many students; 4 = collected for all 
students). Cronbach alpha = .68. 


Academic progress 


Single item. (I A 10) Reflects extent to which institutions collect data on current 
student’s academic progress (1 = not collected; 2 = collected for some students; 3 
= collected for many students; 4 = collected for all students). 



Variable 


Definition 


Post-college assessment 


Three item factorially-derived scale. (I A 1 1.12.14) Reflects the extent to which 
institutions collect data from former students: vocational or professional 
outcomes; further education; satisfaction and experiences with institution after 
leaving (1 = not collected: 2 = collected for some students; 3 = collected for many 
students: 4 = collected for all students). Cronbach alpha = S3. 


Civic/social roles 


Single item. (I A 13) Reflects extent to which institutions collect data on former 
student's civic or social roles in the community (1 = not collected: 2 = collected 
for some students: 3 = collected for many students; 4 = collected tor all students). 


Timing of data collection 


Nine item additive index. (I A 1-9) Reflects when institutions collect data (1 =. 
not collected: 2 = collected at one point in time: 3 = collected at entry and while 
enrolled, or while enrolled and at exit; 4 = collected at entry and at exit; 5 = 
collected at entry, while enrolled, and at exit). 


Number of instruments 


Nine item additive index. (I B 1-9) Reflects student assessment instruments 
(institutionally developed, state provided, and commercially available) used by 
institution to collect ten types of assessment information: student plans or 
expectations: basic college-readiness skills: higher-order cognitive skills: general 
education competencies; competence in major; vocational or professional skills; 
personal growth and affective development; experiences or involvement with 
institution; satisfaction with institution (1 = instrument used; 0 = instrument not 
used). 


Student-centered methods 


Four item factorially-derived scale. (I C 1-4) Reflects the extent to which 
institutions use innovative or nontraditional assessment methods: performance in 
capstone courses; portfolios or comprehensive projects: observations of student 
performance; individual interviews or focus groups ( 1 = not used: 2 = used in 
some units: 3 = used in most units: 4 = used in all units). Cronbach alpha = .61 . 


External methods 


Two item factorially-derived scale. (I C 8-9) Reflects the extent to which 
institutions use assessment methods that data from external constituencies: 
employer interviews or focus groups; alumni interviews or focus groups (1 = not 
used; 2 = used in some units: 3 = used in most units: 4 = used in all units). 
Cronbach alpha = .63. 


Transcript analysis 


Single item. (I C 5) Reflects extent to which institutions use transcript analysis 
to collect student assessment information (1 = not used; 2 = used in some units: 
3 = used in most units; 4 = used in all units). 


External examination 


Single item. (I C 6) Reflects extent to which institutions use external 
examinations to collect student assessment information (1 = not used; 2 = used in 
some units: 3 = used in most units; 4 = used in all units). 


Interviews of withdrawing 
students 


Single item. (I C 7) Reflects extent to which institutions use interviews with 
withdrawing students to collect student assessment information ( 1 = not used; 2 = 
used in some units; 3 = used in most units; 4 = used in all units). 



Variable 


Definition 


Student sub-populations 


Four single items. (ID 1-4) Reflect the use of different assessment methods for 
the following different student populations: a = adult students: b = part-time 
students; c = minority students; d = distance education students ( 1 = different 
method; 2 = same method). 


Number of studies 


Nine item additive index. (I E 1-9) Reflects the number of studies institutions 
conduct on the relationship between aspects of students* institutional experiences 
and performance: course-taking patterns: exposure to different leaching methods; 
patterns of student-faculty interaction: extra-curricular activities: residence 
arrangements; financial aid and/or employment: admission standards or policies; 
academic advising patterns: classroom, library and/or computing resources ( 1 = 
conduct study; 0 = do not conduct study). 


Number of reports 


Five item additive index. (I F 1-5) Reflects the levels of aggregation at which 
student assessment data are provided as reports: institution-wide: schools or 
colleges; academic programs or departments; special populations or subgroups of 
students; by course or groups of courses ( 1 = report provided; 0 = report not 
provided). 



Organizational and Administrative Support Patterns 

Mission emphasis Three item additive index. (II A ] a-c) Reflects institutions' mission statement 





emphasis on undergraduate education and its assessment: emphasizes excellence in 
undergraduate education; identifies educational outcomes intended for students; 
refers to student assessment as important activity ( 1 = yes; 0 = no). 


Internal purposes 


Four item factorially-derived score. (II B 3-6) Reflects the importance of internal 
institutional purposes for undertaking student assessment: guiding undergraduate 
academic program improvement: improving achievement of undergraduate 
students; improving faculty instructional performance: guiding resource al location 
decisions (1 = no importance; 2 = minor importance: 3 = moderate importance; 4 
= very important). Cronbach alpha = .79. 


Accreditation purposes 


Single item. (II B 1) Reflects importance of preparing for institutional 
accreditation self-study as a purpose for undertaking student assessment (1 = no 
importance; 2 = minor importance: 3 = moderate importance; 4 = very 
important). 


Slate purposes 


Single item. (II B 2) Reflects importance of meeting state reporting requirements 
as a purpose for undertaking student assessment ( 1 = no importance; 2 = minor 
importance; 3 = moderate importance; 4 = very important). 


Administrative and governance 
activities 


Seven item additive index. (II C 1-7) Reflects the number of administrative or 
governance activities used by institutions to promote student assessment: annual 
institution-wide assessment forums or seminars; rewards or incentives for 
administrators promoting use of assessment in unit; incentives for academic units 
to use assessment information; assessment workshops for administrators; board 
of trustees committee addresses assessment; faculty governance committee 
addresses assessment; student representation on assessment committees (1 = yes; 

0 = no). 



Variable 


Definition 


Administrative and faculty 
support 


Four item additive index. (II D 2-5) Reflects the degree to which chief executive 
officer, academic and student affairs administrators, and faculty support student 
assessment (1 = very unsupporlive; 2 = somewhat unsupporlive: 3 = neutral or 
unknown; 4 = somewhat supportive; 5 = very supportive). 


Type of plan or policy 


Seven single items. (II E 1 a-g) Reflects the institutions plan or policy for 
student assessment: a = formally adopted plan or policy requiring assessment 
activities for ah academic units; b = formally adopted plan or policy requiring 
assessment activities for some academic units; c = formally adopted plan or 
policy requiring al] academic units to develop their ow'n assessment plan; d = 
formally adopted plan or policy stipulating instilution-wnde activities to be 
conducted by central committee, office, or officer; e = has no formal plan or 
policy but academic units are encouraged to conduct their own assessment 
activities; f = is currently developing plan or policy; g = does not have an 
assessment plan or policy (I = yes; 0 = no). 


Formal centralized policy 


Single item. (II E 1 a) Reflects institution has formal institutional plan or policy 
requiring specified student assessment activities of all academic units or programs 
( 1 = yes; 0 = no). 


Institution-wide planning 
group 


Single item. (II E 2) Reflects institution has institution-wide group for student 
assessment planning and policy setting ( 1 = ves; 0 = no). 


Breadth of assessment planning 
group 


Nine item additive index. (II E 3 a-i) Reflects the number of internal members 
included in the institution’s assessment planning group: chief executive officer; 
academic affairs administrator(s)/staff; student affairs administrator(s)/slaff; 
institutional research administrator(s)/staff; academic review and evaluation 
administrator(s) /staff; student assessment administralor(s)/siaff; faculty; students; 
other. 


Responsibility for planning 
group 


Seven single items. (II E 4 a-g) Reflect the internal members w'ho have executive 
responsibility for the institution-wide group responsible for planning or policy- 
setting for assessment: a = academic affairs administrator; b = student affairs 
administrator; c = institutional research officer; d = academic review' and 
evaluation officer; e = student assessment officer; f = faculty member; g = other 
( 1 = yes; 0 = no). 


Approval authority 


Eleven single items. (II E 5 a-k) Reflect the internal members who approve any 
changes to institutions assessment plan or policy: a = board of trustees; b = chief 
executive officer; c = chief academic affairs officer; d = chief student affairs officer; 
e = institutional research officer; f = academic review and evaluation officer; g = 
student assessment officer; h = student government; i = academic senate or other 
faculty committees: j = faculty union; k = other ( 1 = yes; 0 = no). 


Operating responsibility 


Eight single items. (II E 6 a-h) Reflect the internal members who have 
operational responsibility for the institution’s day-to-day assessment activities: a 
= academic affairs administrator; b = student affairs administrator; c = institutional 
research officer; d = academic review and evaluation officer; e = student 
assessment officer; f = faculty member; g = other; h = no one (1 = yes; 0 = no). 



Variable 


Definition 


Reporting relationship 


Six single items. (II E 7 a-'f) Reflect the individual to whom person with day-to- 
day responsibility reports: a = chief executive officer: b = chief academic affairs 
officer; c = chief student affairs officer; d = institutional research officer: e = 
academic review and evaluation officer: f = other ( I = yes; 0 = no). 


Conducted evaluation 


Single item. (II F 1 a-d) Reflects if institution has formally or informally 
evaluated its student assessment process (1 = yes; 0 = no). 


Evaluations elements 


Eight single items. (II F 2 a-h) Reflect the elements that were reviewed during the 
institutions assessment evaluation: a = student assessment plan or policies: b = 
structure and responsibility for assessment; c = achievement of intended 
objectives; d = reliability and validity of instruments and methods: e = quality of 
data analysis; f = use of information in institutional decision-making; g = 
problems encountered: h = comparison of costs and benefits (1 = yes: 0 = no). 



Assessment Management Policies and Practices 

Budget decisions Two item additive index. (IV A 3-4) Reflects fomial use of assessment 





information in the budget process: to competitively allocate resources among 
academic units; to reward academic units for improvement ( 1 = yes: 0= no). 


Computer support 


Three item additive index. (IV B 2-4) Reflects institutional capacity to collect and 
manage student assessment information: computerized student information 
system includes student performance indicators; student information system tracks 
individual students; student assessment database integrated with other institutional 
databases ( 1 = yes: 0 = no). 


Access to information 


Five item additive index. (IV C 1-5) Reflects internal accessibility of assessment 
information on individual students by: institutional research or assessment 
professionals; senior academic administrators; department chairs or academic 
program administrators; student affairs professionals: faculty advisors (1 = yes: 0 
= no). 


Distribution of reports 


Six item additive index. (IV D 1-6) Reflects the number of constituent groups to 
whom student assessment reports are regularly distributed: students; faculty; 
academic administrators; student affairs professionals; employers; general public 
( 1 = yes; 0 = no). 


Student involvement 


Three item factorially-derived scale. (IV E 1 .3,4) Reflects the extent to which 
institutions have policies or practices to promote student involvement in 
assessment activities: inform students about assessment purposes and uses; 
require students to participate in assessment activities; provide students with 
individual feedback on assessment results (1 = not done at all; 2 = done in a few 
departments; 3 = done in some departments; 4 = done in many departments; 5 = 
done in most departments). Cronbach alpha = .69. 
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Variable 

Professional development 



Student affairs training 



Definition 

Four item factorially-derived scale. (IV F 2-5). Reflects existence of professional 
development policies or practices on student assessment for faculty and academic 
administrators: provide funds for faculty to attend or present at assessment 
conferences: offer student assessment workshops or consultation for faculty; 
provide assistance (e.g.. paid leaves, stipends, course reduction ) to improve 
faculty use of student assessment: provide student assessment workshops for 
academic administrators (1 = not done at all: 2 = done in a few departments; 3 = 
done in some departments: 4 = done in many departments; 5 = done in most 
departments). Cronhach alpha = .77. 

Two item factorially-derived scale. (IV F 6-7) Reflects existence of professional 
development policies or practices on student assessment for student affairs 
personnel: require assessment training for student affairs staff; provide student 
assessment workshops for student affairs administrators ( 1 = not done at all; 2 = 
done in a few departments; 3 = done in some departments: 4 = done in many 
departments; 5 = done in most departments). Cronhach alpha = .84. 



Faculty evaluation Five item factorially-derived scale. (IV G 1-5) Reflects existence of faculty 

evaluation and reward policies and practices related to student assessment: 
promotion evaluation considers evidence of student performance: salary evaluation 
considers evidence of student performance; promotion, tenure or salary reviews 
consider faculty participation in student assessment; promotion, tenure or salary 
reviews consider scholarship on assessment; public recognition or awards for 
faculty use of student assessment ( 1 = not done at all; 2 = done in a few 
departments; 3 = done in some departments; 4 = done in many departments; 5 = 
done in most departments). Cronhach alpha = .77. 

Academic planning and review Four item factorially-derived scale. (IV H 1-4) Reflects the incorporation of 

student assessment data into academic planning and review processes for: 
academic departments or undergraduate programs: general education or core 
curriculum; courses; student academic support services (1 = not done at all: 2 = 
done in a few departments; 3 = done in some departments; 4 = done in many 
departments; 5 = done in most departments). Cronhach alpha = .84. 



Institutional Uses of Student Assessment 

Educational decisions Ten item factorially-derived scale. (V A 1-5, 8-12) Reflects the influence of 

student assessment information in educational decisions: revision of undergraduate 
academic mission or goals; designing or reorganizing academic programs or 
majors: designing or reorganizing student affairs units; allocating resources to 
academic units; modifying student assessment plans, policies, or processes; 
revising or modifying general education curriculum; creating or modifying student 
out-of-class learning experiences; creating or modifying distance learning 
initiatives; modifying instructional or teaching methods; modifying student 
academic support services (I = no action or influence known; 2 = action taken, 
data not influential; 3 = action taken, data somewhat influential; 4 = action taken, 
data very influential), Cronhach alpha = .83, 
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Variable 

Faculty decisions 



Definition 

Two item factorially-derived scale. (V A 6-7) Reflects the influence of student 
assessment information in faculty decisions: deciding faculty promotion and 
tenure: deciding faculty salary increases or rewards (1 = no action or influence 
known: 2 = action taken, data not influential: 3 = action taken, data somewhat 
influential: 4 = action taken, data very influential). Cronbach alpha = .79. 



Institutional Impacts of Student Assessment 



Faculty impacts 



Student impacts 



External impacts 



Four item factorially-derived scale. (V B 1-4) Reflects student assessment impacts 
on faculty: affected campus discussions of undergraduate education: contributed to 
faculty satisfaction; contributed to faculty interest in leaching; led to changes in 
teaching methods used (I = not monitored, do not know; 2 = monitored, negative 
impact; 3 = monitored, no known impact; 4 = monitored, positive impact). 
Cronbach alpha = .79. 

Four item factorially-derived scale. (V B 5-8) Reflects student assessment impacts 
on students: contributed to student satisfaction: affected student retention or 
graduation rates: affected student grade performance; affected student achievement 
on external examinations (I = not monitored, do not know: 2 = monitored, 
negative impact; 3 = monitored, no known impact: 4 = monitored, positive 
impact). Cronbach alpha = .82. 

Seven item factorially-derived scale. (V B 9-15) Reflects student assessment 
impacts on external constituents: affected student applications or acceptance rates; 
affected allocation or share of state funding; affected evaluation from regional 
accrediting agency; affected private fund-raising results; affected success on grant 
applications; affected communications with external constituents; affected 
institutional reputation or image (1 = not monitored, do not know; 2 = 
monitored, negative impact; 3 = monitored, no known impact; 4 = monitored, 
positive impact). Cronbach alpha = .82. 



1 Assessment of Teaching and Learning for Improvement and Public Accountability: State Governing, Coordinating 
Board and Regional Accreditation Association Policies and Practices (Cole et al., 1997) 
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